Ettevalmistustoetuse taotlemine (ERC)

5868670 428625 0 0 -3810 9839325 Ehitajate tee 5 19086 Tallinn Rg-kood 74000323 Tel 620 2002 E-post [email protected] www.taltech.ee 0 0 Ehitajate tee 5 19086 Tallinn Rg-kood 74000323 Tel 620 2002 E-post [email protected] www.taltech.ee -3810 428625 0 0 SA Eesti Teadusagentuur Soola 8 28 . 11 .20 2 3 nr 11-10/ 575- 8 51004 Tartu Ettevalmistustoetuse taotlemine Käesolevaga taotleb Tallinna Tehnikaülikool ( reg nr 74000323, ak EE201010052037382001 ) ettevalmistustoetust järgmistele ERC p rojekti taotlustele: Theory of More-Than-Human Management - M-BOT ( 101123699 , ERC-2023-COG ) , p ro f Mari-Klara Stein, ärikorralduse instituut; Digital super telescopes and RIGOrous theory of Radars – RIGOR ( 101118952 , ERC-2023-SyG ), prof Lassi Päivärinta , küberneetika instituut . Lugupidamisega (allkirjastatud digitaalselt) Marika Lunden Teadusosakond Lisa: ESR id left 284480 Riina Vilgats 620 3536 [email protected] 0 0 Riina Vilgats 620 3536 [email protected] Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 1, 30102023] Step 2 Evaluation Report CONFIDENTIAL Call reference ERC-2023-COG Activity HORIZON ERC Grants Funding scheme ERC CONSOLIDATOR GRANTS Panel name SH1 Proposal No. 101123699 Acronym M-BOT Applicant Name Mari-Klara STEIN Title More-Than-Human Management: “Bot-ification” and Beyond EVALUATION CRITERIA Criterion 1 - RESEARCH PROJECT Ground-breaking nature and potential impact of the research project To what extent does the proposed research address important challenges? To what extent are the objectives ambitious and beyond the state of the art (e.g. novel concepts and approaches or development between or across disciplines)? To what extent is the proposed research high risk-high gain (i.e. if successful the payoffs will be very significant, but there is a high risk that the research project does not entirely fulfil its aims)? Scientific Approach To what extent is the outlined scientific approach feasible bearing in mind the extent that the proposed research is high risk-high gain? To what extent are the proposed research methodology and working arrangements appropriate to achieve the goals of the project? To what extent does the proposal involve the development of novel methodology? To what extent are the proposed timescales, resources and PI commitment adequate and properly justified? Criterion 2 - PRINCIPAL INVESTIGATOR Intellectual capacity and creativity The questions below can have one of the following five responses: Exceptional/Excellent/Very Good/Good/Non-competitive To what extent has the PI demonstrated the ability to conduct ground-breaking research? To what extent does the PI provide evidence of creative independent thinking? To what extent does the PI have the required scientific expertise and capacity to successfully execute the project? PANEL SCORE AND RANKING RANGE Final panel score: B (meets some but not all elements of Ranking range*: 80%-100% the ERC's excellence criterion and will For your information, only the top 42% of the proposals not be funded) evaluated in panel SH1 in Step 2 were funded. * Ranking range of your proposal out of the proposals evaluated by the panel in Step 2, in percent, from 1% for the highest ranked proposals to 100% for the lowest ranked. 101123699 / 30.10.2023 Page 1 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 2, 30102023] PANEL COMMENT This evaluation report contains the final recommendations and score awarded by the ERC review panel during the second step of the ERC Consolidator Grant review and the ranking range. The discussion of the panel was conducted within the context of prior reviews submitted by ERC panel members and external referees and the interview with the applicant. The panel closely examined all the individual review reports and, while not necessarily subscribing to each and every opinion expressed, found that they provide a fair overall assessment. The comments of the individual reviewers are included in this report. The presentation given by the applicant during the interview and the answers to the questions that were addressed greatly contributed to build the panel's view about the proposal's strengths and weaknesses. Both the individual reviews and the interview were the basis for the discussion and the final recommendation of the panel. This proposal explores the collaboration between smart machines and human managers and examines the kind of managerial work that emerges from such cooperation. To achieve this, the project presents three working hypotheses, each focusing on a specific scenario in which human managers and machines work together: (1) delegation, (2) sequential co-work, and (3) aggregated co-work. The panel appreciated the candidate’s extensive track record in the field of digital transformation, particularly in relation to its impact on employees, organizations, and managers. The panel noted the ground-breaking potential of the proposal but concluded that the relationship between the qualitative (interviews) and the quantitative (ABM) parts was not entirely clear. The information and explanations provided by the candidate during the interview did not add sufficient clarity. The panel therefore does not recommend the proposal for funding. 101123699 / 30.10.2023 Page 2 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 3, 30102023] REVIEWER COMMENTS The following individual reviews have been carried out independently prior to the panel meeting and do not necessarily reflect the panel's final opinion Reviewer 1 Research Project Ground-breaking nature and potential impact of the research project The project is structured around a comprehensive mixed-methods approach, encompassing three distinct phases: hypotheses development, hypotheses testing, and theory development. In the initial phase of hypothesis development, the primary focus is on addressing research question #1, which delves into the dynamics of collaboration between smart machines and human managers. This is excellent. This phase aims to meticulously formulate working hypotheses denoted as A, B, and C, which collectively aim to provide a preliminary framework for addressing research question #2. Nevertheless, the set goals lack a degree of ambition and fail to transcend the existing state of knowledge, either through the introduction of innovative concepts and methodologies or by facilitating cross-disciplinary advancements. This absence of heightened ambition stands as a notable drawback. Furthermore, inherent constraints pertaining to how attributes and behavioral regulations can be precisely outlined within the context of Agent-Based Modeling (ABM) might lead to a dilution of novelty during the translation of qualitative insights into empirically testable hypotheses, thereby posing a risk of generating inconclusive outcomes. This limitation represents a significant concern. Scientific Approach The project outlined a feasible scientific approach using mixed methods, a three-phase approach, and (1) qualitative case study research to develop hypotheses, (2) agent-based modeling to test those hypotheses, and (3) substantive and abstract mechanism-based explanations to develop theories. Due to the possibility that full delegation, sequential co-work, and aggregated co-work may prevent the hypotheses from being evaluated in a field context, the proposed research is high risk–high benefit. The proposed timescales, resources and PI commitment is adequate and it is properly justified. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Very good To what extent does the PI provide evidence of creative independent thinking? Very good To what extent does the PI have the required scientific expertise and capacity to successfully Very good execute the project? Comments (Optional for reviewers) The proposal is notably founded upon the scaffolding of prior research endeavors, thereby indicating a well-defined trajectory of scholarly progression. Furthermore, the existence of a well-structured pipeline for forthcoming publications contributes to the comprehensive scholarly standing of the PI. 101123699 / 30.10.2023 Page 3 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 4, 30102023] Reviewer 2 Research Project Ground-breaking nature and potential impact of the research project This project seeks to shed light on the work between humans and machines and the types of managerial work this human-machine interaction generates. This is an important topic for management as machines are increasingly used to make or contribute to managerial decisions. However, the ground-breaking potential of the project is not clear from reading the proposal. The focus is put on three types of "ideal" interactions, with either (i) decision-making delegated to machines, or (ii) taking place sequentially between humans and machines, or (iii) with humans and machines working independently with the final decision aggregating inputs from both. The research proposal contains an extensive description of existing literature, which serves the PI to derive working hypotheses. The statements of those hypotheses, although central to the research proposal, are not self-explanatory. For instance, the terms “more-than-human managerial work” and “augmented system” are repeatedly used without being described. The PI acknowledges that there are few real-world examples fitting the types of interactions considered – which poses challenges for testing the hypothesis in real-life scenarios. The PI then proposes to rely on an approach she is not yet familiar with (“The applicant is in the process of gaining proficiency with this method”), namely, an agent-based model. As explained below, it is unclear whether this approach is able to provide empirical regularities to address the research questions outlined in the proposal. It is also unclear how the outcomes of the approach could deliver lessons to improve the interactions between humans and machines. Scientific Approach The research approach combines “qualitative exploratory research” through case studies, built mainly around interviews with managers and analysis of log data generated by algorithms. Instead of testing the hypotheses in similar settings as those from where they are obtained, the PI proposes to use an agent-based model. Simulations with programmed behavior rules drive this model. It is thus unclear whether the outcome will just reflect those rules or the actual behavior that would be expected from true human-machine interactions. The PI mentions a theory development phase, which does not seem to be a formal theory. It is based on “reading, reflecting, and writing,” following an existing approach (Rivard, S. (2021), published in the Journal of Information Technology). Overall, the project does not contain a relevant methodological contribution. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Good To what extent does the PI provide evidence of creative independent thinking? Good To what extent does the PI have the required scientific expertise and capacity to successfully Good execute the project? Comments (Optional for reviewers) The PI's research has mostly been focused on analyzing the effects of technology adoption in the workplace, which is closely linked to the current research proposal. She has a publication record covering this topic in various journals in management, such as the Journal of Information Technology or Journal of Behavioral Decision Making, among others, none of which are considered to be top-field journals in Management. The PI has extensive experience in conducting the qualitative approach included in the proposal but is not trained yet to develop the agent-based model, which is central to her proposal. 101123699 / 30.10.2023 Page 4 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 5, 30102023] Reviewer 3 Research Project Ground-breaking nature and potential impact of the research project The proposal engages with the urgent and complex challenges facing modern management, particularly the integration of smart machines and AI. Its ambitious aim to develop new theoretical frameworks in human-AI collaboration is compelling. The project promises transformative impacts on both theory and practice, should it succeed. The proposal would nevertheless benefit from a deeper engagement with the existing body of research related to algorithms and AI in the management field to ensure going beyond the current state of knowledge. Such resources include, but are not limited to: - Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021). Managing artificial intelligence. MIS Quarterly, 45(3). - Faraj, S., Pachidi, S., & Sayegh, K. (2018). Working and organizing in the age of the learning algorithm. Information and Organization, 28(1), 62-70. - Jarrahi, M. H. (2018). Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making. Business Horizons, 61(4), 577-586. - Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation–augmentation paradox. Academy of Management Review, 46(1), 192-210. Scientific Approach The proposed methods fit the research aims. The timeline provides logical progression and budgeting appears justified. The PI has relevant expertise to guide execution and appropriate risks are acknowledged. Incorporating more behavioral data from the field in the second project stage could enrich the model development, model validation, and mitigate some of the challenges in accurately modelling complex organizational and cognitive processes through simulations. A clear challenge is the limited expertise in computer science, statistics, human-computer interaction, or related technical fields (including simulations, as acknowledged in the proposal). This is not least the case given the complexity of developing, understanding, and deploying algorithmic systems. Overall, the approach is still reasonably designed to achieve the ambitions. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Exceptional To what extent does the PI provide evidence of creative independent thinking? Excellent To what extent does the PI have the required scientific expertise and capacity to successfully Excellent execute the project? Comments (Optional for reviewers) No comments received 101123699 / 30.10.2023 Page 5 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 6, 30102023] Reviewer 4 Research Project Ground-breaking nature and potential impact of the research project To what extent does the proposed research address important challenges? The research focus is very timely and topical: we have seen the sharp rise of AI, machine learning and natural language processing, including generative AI recently, which poses many questions and challenges for organizations, workers and managers. The concept of ‘more-than-human management’ is proposed as likely to differ substantially from traditional human management. To what extent are the objectives ambitious and beyond the state of the art (e.g., novel concepts and approaches or development between or across disciplines)? This is a very ambitious proposal which aims to address difficult if not intractable issues. The nature of ‘human’ managerial work is extremely complex, and investigating how human managers can work with ‘smart’ machines is a vast enterprise. And, as rightly stated in the proposal, “little is known (…) about what managerial work routines will actually be like when practiced by humans and smart machines together”. The aim is therefore to build a new ‘middle- range’ theory of ‘more-than-human’ management. Theoretically, the proposal essentially draws on the existing small number of studies which have investigated how human managers and smart machines can work together; these are largely situated in the discipline of management information systems research, and more precisely in the novel field of ‘algorithmic management’; presumably the notion of ‘more-than-human management’ is offered here to contrast with and expand upon ‘algorithmic management’. There is some, although fairly limited, overlap with literature in organizational behaviour on the nature of managerial work. I am not entirely clear where this project would contribute in that area, apart from renewing attention on managerial competencies (as mentioned in Table 2 Immediate Impact) in the context of managers using smart machines. This would likely fall within the practical field of human resource management, especially training. The intended ‘middle-range’ theorizing is not developed in the proposal, so I cannot comment on how this theorizing would contribute to organization studies in general and to the understanding of managerial work in particular. Specifically, the proposal states that it will establish an ‘entirely new stream of research’ about how full collaboration between managers and smart machines ‘will gain in scope of activities and a gain in depth of knowledge of the activities, including gain in professional judgement, that are part of managerial work’ – this is with regards to the third hypothesis C. I am not sure how one determines a ‘gain’ in this context and how this could relate to established organizational studies of managerial work. Why and how do the ‘emerging characteristics (e.g., scope of activities, depth of knowledge, uniqueness of knowledge)’ describe ‘the quality of managerial work’ when compared to extent research? Much research has exposed the difficulties in determining such ‘gains’ (who for? Compared to what?) and ‘quality’ (from what standpoint?) because they are highly dependent on the organizational context. Additionally, the first empirical phase includes the gathering of data about workers, but this is not mentioned or theorised in the State-of-the- Art section. To what extent is the proposed research high risk-high gain (i.e., if successful the payoffs will be very significant, but there is a high risk that the research project does not entirely fulfil its aims)? The proposal draws on three categories developed in the literature on ‘algorithmic management’: delegation, sequential co-work and aggregated co-work. These categories range respectively from a strong to a weak control of managerial work by the smart machines, or put another way, a weak to a strong collaboration between managers and machines. The research project intends to assess how these three categories relate to levels of scope/variety of activities, depth and uniqueness of knowledge, professional judgement, wisdom of decisions, and skills in managerial work. Intuitively, it seems likely that these three categories range from low levels or ‘bot-ified’ managerial work to higher levels or a richer kind of managerial work respectively. This is reflected in the three hypotheses proposed. One would assume that 101123699 / 30.10.2023 Page 6 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 7, 30102023] this can be demonstrated empirically fairly easily and can have practical managerial implications; however, I am not sure how theoretically ground-breaking that is outside the subfield of algorithmic management, and whether this will reach the ambitious aim of an ‘entirely new stream of research’. Scientific Approach To what extent is the outlined scientific approach feasible bearing in mind the extent that the proposed research is high risk-high gain? The first empirical phase with the three qualitative case studies follows logically from the three hypotheses and appears feasible; and exploring workers’ perspectives (as well as managers’) is a good idea, although it is not referred to or conceptualized elsewhere in the proposal. In terms of what data will be gathered, it is mainly about their perceptions about the nature and quality of managerial work. This should be emphasized earlier in the proposal. Furthermore, for hypothesis C, ‘gain’ implies gain over, or compared to, something else (prior managerial work pre-smart machine, I presume). It is unclear how this will be elicited beyond opinions and perceptions. To what extent are the proposed research methodology and working arrangements appropriate to achieve the goals of the project? In the first empirical phase, the range of primary data gathering methods is interesting and again feasible if good company access is afforded, which seems to be the case, and there is a strong effort to ensure alternative empirical sites in case the initial ones do not work out. The data collection techniques and coding for the first phase are well described. The sampling of case studies is well justified in view of the three categories and three hypotheses mentioned above, upon which the first phase is clearly based. The risk for the third company case (corresponding to the third hypothesis) is clearly signaled, this kind of ‘augmented’ managerial work still being rare in practice. The second experimental phase is well justified in the context of the study and due to the practical difficulties in comparing the three scenarios in situ. It would have been helpful to provide a better rationale for the choice of agent- based modelling compared to alternatives. This section is also lacking details on the experimental design: which ‘variables’ will be operationalized, and how do these variables relate to the preceding phase, conceptually? What will the ‘causal outcomes’ be? How will they be assessed? Which managerial tasks will be simulated, tested, how, with whom, how many, how long for, etc.? The last phase and section on ‘Theory development’ could have been much shorter, Figure 1 is self-explanatory. It would have then given space for providing a lot more details about the preceding experimental phase. To what extent does the proposal involve the development of novel methodology? The mixed methods methodology is interesting and there has been many calls for more usage of mixed methods in the field of information systems and management research in general. The particular mix here is original. However, the links between the first qualitative and the second experimental phase should be made clearer conceptually so that we have a much stronger sense of which type of theorizing will be developed in the final phase. To what extent are the proposed timescales, resources and PI commitment adequate and properly justified? These are well thought out and seem well justified, particularly the reliance on an external expert in agent-based modelling which appears to be the PI’s main weakness. The PI is clearly well versed in qualitative research, and this is a fairly radical change of direction. Comments: I am not entirely convinced about the second phase of the research design, and complementing it with an external expert is a risky strategy; it may not ensure knowledge transfer to the PI and doctoral and post-doctoral team, which is crucial in such an ambitious project. 101123699 / 30.10.2023 Page 7 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 8, 30102023] Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Very good To what extent does the PI provide evidence of creative independent thinking? Excellent To what extent does the PI have the required scientific expertise and capacity to successfully Very good execute the project? Comments (Optional for reviewers) The PI is extremely competent for the first qualitative research phase, but she is moving into unchartered waters by embarking in a complex experimental second phase, which presents risks. Combining both is also a challenge and this proposal does not quite convince me that it can be met. 101123699 / 30.10.2023 Page 8 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 9, 30102023] Reviewer 5 Research Project Ground-breaking nature and potential impact of the research project The project examines (1) how smart machines and human managers work together, and (2) what kind of managerial work (e.g., how rich in skills, how wise in decisions) this generates, with the aim of building a new general theory of more-than-human management. To this end, the project puts forward 3 working hypotheses each focusing on a specific scenario in which human managers and machines work together, namely (1) delegation, (2) sequential co- work, and (3) aggregated co-work. The overall theme of the project is obviously highly topical and relevant, as algorithms are increasingly taking over activities traditionally taken care of by humans. Management is a good example of this development. Altogether, the project is well-positioned to contribute to the scholarly discussion on the future of work taking place within the information systems (IS) and management and organization (M&O) disciplines. However, the research plan does not explicitly articulate what the PI considers the project's main sources of academic breakthroughs. My interpretation is that the main potential breakthrough could emerge from the theory development phase of the project. It is very difficult, maybe impossible, to meaninfully foresee at this point what kind of theory will emerge from the project and whether this . Hence, the project fulfills the criterion of being high-risk. Scientific Approach The methods presented support meeting the objectives of the project. The PI clearly has a very strong track record and competence in conducting qualitative research. The agent-based modeling to be undertaken in the second phase of the projec can be consider a novel methodology in the scholarly discourse(s) the project is positioned to contribute to. The method can potentially be relevant to meet the objectives of the project. At the same time, however, the PI is in the process of building her competence and command of the method. This means that the project is quite heavily reliant on collaborations in this area. I don't consider this a problem per se, but it is a potential risk regarding the execution of the research activities. The present scheduling and resourcing seem to appropriate. The PI meaningfully acknowledges the host institution's (understandable) challenges related to recruiting PhD students. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Exceptional To what extent does the PI provide evidence of creative independent thinking? Exceptional To what extent does the PI have the required scientific expertise and capacity to successfully Very good execute the project? Comments (Optional for reviewers) The PI has a track-record of conducting research related to the topic of the project. She published her work in very good outlets. All in all, the PI has a strong capacity to successfully execute the research. Her track-record in terms of obtaining research funding and leading research projects is however limited. The most relevant merit in this area is the 99k€ funding from Estonian Research Council. 101123699 / 30.10.2023 Page 9 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 10, 30102023] Reviewer 6 Research Project Ground-breaking nature and potential impact of the research project The proposed research addresses the new phenomenon of smart machines and human managers working together and the new output and work they generate. The research objectives are ambitious in terms of providing insights into new ways of working at a managerial level, supported by intelligent artificial systems. The high-gain element of the research is its relevance to organisations' current state of play in the wake of digitisation and artificial intelligence applications. Until recently, there has been a noticeable lack of emphasis within the management field on the digitalization of managerial decisions and activities. This proposal has the potential to address some of the existing gaps in the literature by introducing a novel concept termed 'more than human management.' This concept indicates the opportunity to integrate new theoretical elements for the creation and examination of innovative theories in this particular domain. Scientific Approach Although this project is characterized by its methodological complexity and the intricate nature of its subject matter, it acknowledges and addresses relevant risks, demonstrating a robust approach to mitigating potential challenges. The project proposes a scientific approach that utilizes mixed methods in a three-phase strategy, involving developing hypotheses through qualitative analysis, testing the hypotheses through agent-based modelling, and developing theories. It is worth noting that the research carries a high-risk, high-reward element due to the potential inability to test hypotheses in a real-world setting. This proposal establishes a set of operational hypotheses that enable the examination of managerial work beyond human-centric perspectives under various hypothetical scenarios, with documented involvement from two of the three case studies. The choice of employing agent-based modelling is well-founded, considering that the identified hypotheses may not be testable in practical field settings. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Excellent To what extent does the PI provide evidence of creative independent thinking? Excellent To what extent does the PI have the required scientific expertise and capacity to successfully Excellent execute the project? Comments (Optional for reviewers) The PI has showcased pioneering research abilities while collaborating with doctoral and post-doctoral scholars. The budget is thoughtfully allocated towards a substantial collaborative research team. This proposal is an extension of the PI's prior research efforts and leverages an existing stream of publications. The PI possesses a deep comprehension of digital transformation and its impact on work and extensive proficiency in qualitative research techniques within this domain. The researcher has significant expertise in conducting influential research related to digital transformation in organisations. 101123699 / 30.10.2023 Page 10 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 11, 30102023] Reviewer 7 Research Project Ground-breaking nature and potential impact of the research project The proposed research tackles an increasingly important but rather under-developed topic in management. While traditional approaches have either relied on human managers or delegated simple tasks to ML algorithms, this project aims to explore the implications of partially automating more complex managerial functions through human and non- human managerial cooperation. This shift from an "either/or dichotomy" to a collaborative approach between human managers and AI introduces a novel perspective and has the potential to shed light on the required managerial skills and decision quality in this evolving landscape, thus contributing to new theory development. Given its unexplored nature, the proposal is classified as a high-risk/high-gain endeavor. One significant risk is the current limitation in studying real-life applications of ML algorithms, as acknowledged by the PI. However, the groundbreaking potential of this research lies in developing new insights into the fast-paced adoption and use of AI in management. By examining how this adoption affects managerial work and interactions with smart machines, the project can shed light on the potential impact on collaboration and decision-making quality. While the research poses certain challenges, the potential gains are substantial. The findings could have far-reaching implications for understanding the changing dynamics in managerial roles, AI-human collaboration, and the optimization of decision-making processes. In conclusion, the proposal presents a high-risk/high-gain research project that addresses a novel and critical aspect of AI adoption in management. By investigating the implications of human and non-human managerial cooperation, the research has the potential to make significant contributions to both theory and practice, offering valuable insights into the future of AI-augmented management processes. Scientific Approach The outlined approach of the proposal is classified as high risk/high gain. It involves the application of mixed methods to develop and test hypotheses, leading to the development of new theory in the field. The research will explore three situations of 'more-than-human' AI management applications, specifically delegation, sequential co-work, and aggregated co-work, utilizing Agent-Based Modeling (ABM) for analysis. One potential risk is the lack of previous experience from the PI in using ABM. While the PI plans to collaborate with an ABM expert and invest effort in gaining proficiency with ABM, the dependence on this method still poses a high risk for the project's success. The use of a new method requires a learning curve and may lead to challenges in implementing the research effectively. The detailed description of the work plan and efforts to mitigate risks is commendable, indicating a well-thought-out strategy for project execution. The justified time plan adds credibility to the proposal's feasibility. A significant portion of the budget is allocated to manpower, including involving another senior scholar with documented ABM experiences. The inclusion of two doctoral students and two postdocs further supports the project's complexity and potential impact. However, it is noted that higher PI involvement would have enhanced the proposal's persuasiveness. Leadership responsibility is crucial for guiding the project and ensuring its successful implementation. A more substantial commitment from the PI may have instilled greater confidence in the project's feasibility. In conclusion, the proposal presents an ambitious and high-risk/high-gain research project exploring 'more-than-human' AI management applications. The mixed methods approach and utilization of ABM introduce both opportunities and challenges. While the detailed work plan and budget allocation are well-justified, the project's dependency on ABM and the importance of higher PI involvement should be considered in addressing potential risks and ensuring successful project execution. Principal Investigator To what extent has the PI demonstrated the ability to conduct ground-breaking research? Excellent To what extent does the PI provide evidence of creative independent thinking? Excellent To what extent does the PI have the required scientific expertise and capacity to successfully Excellent execute the project? Comments (Optional for reviewers) 101123699 / 30.10.2023 Page 11 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)7394518 - 30/10/2023 [Panel: SH1, Page 12, 30102023] This PI’s consistent and high-impact track record in the field of digital transformation, particularly in relation to its impact on employees, organizations, and managers, is a significant asset that clearly support the viability of the proposed research 101123699 / 30.10.2023 Page 12 of 12 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Digitally sealed by the European Commission Date: 2023.10.30 18:36:43 CET This electronic receipt is a digitally signed version of the document submitted by your organisation. Both the content of the document and a set of metadata have been digitally sealed. This digital signature mechanism, using a public-private key pair mechanism, uniquely binds this eReceipt to the modules of the Funding & Tenders Portal of the European Commission, to the transaction for which it was generated and ensures its full integrity. Therefore a complete digitally signed trail of the transaction is available both for your organisation and for the issuer of the eReceipt. Any attempt to modify the content will lead to a break of the integrity of the electronic signature, which can be verified at any time by clicking on the eReceipt validation symbol. More info about eReceipts can be found in the FAQ page of the Funding & Tenders Portal. (https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq) Commission européenne/Europese Commissie, 1049 Bruxelles/Brussel, BELGIQUE/BELGIË - Tel. +32 22991111  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 1, 14072023] Step 2 Evaluation Report CONFIDENTIAL Call reference ERC-2023-SyG Activity HORIZON ERC Synergy Grants Funding scheme ERC Synergy Grants Panel name SyG2D Proposal No. 101118952 Acronym RIGOR Applicant(s) Lassi Juhani PÄIVÄRINTA; Michael FLETCHER; Markku LEHTINEN Title Digital super telescopes and RIGOrous theory of Radars EVALUATION CRITERIA Criterion 1 - RESEARCH PROJECT Ground-breaking nature and potential impact of the research project To what extent does the proposed research address important challenges? To what extent are the objectives ambitious and beyond the state of the art (e.g. novel concepts and approaches or development between or across disciplines)? To what extent is the proposed research high risk-high gain (i.e. if successful the payoffs will be very significant, but there is a high risk that the research project does not entirely fulfil its aims)? Scientific Approach To what extent is the outlined scientific approach feasible bearing in mind the extent that the proposed research is high risk- high gain? To what extent does the proposal go beyond what the individual Principal Investigators could achieve alone? To what extent do the Principal Investigators succeed in proposing a combination of scientific approaches that are crucial to address the scope and complexity of the research question to be tackled? To what extent are the proposed research methodology and working arrangements appropriate to achieve the goals of the project? To what extent does the proposal involve the development of novel methodology? To what extent are the proposed timescales, resources and PI commitment adequate and properly justified? Criterion 2 - PRINCIPAL INVESTIGATORS Intellectual capacity and creativity The questions below can have one of the following five responses: Exceptional/Excellent/Very Good/Good/Non-competitive To what extent have the PIs demonstrated the ability to conduct ground-breaking research? To what extent do the PIs have the required scientific expertise and capacity to successfully execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - necessary to address the proposed research question? PANEL SCORE AND RANKING RANGE Final panel score: B (is of high quality but not sufficient to Ranking range*: 58%-100% pass to Step 3 of the evaluation. Please For your information, only the top 57% of the proposals note that you may also be subject to evaluated in panel SyG2D were retained for Step 3. resubmission limitations in the next call) * Ranking range of your proposal out of the proposals evaluated by the panel in Step 2, in percent, from 1% for the highest ranked proposals to 100% for the lowest ranked. 101118952 / 14.07.2023 Page 1 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 2, 14072023] PANEL COMMENT This evaluation report contains the final scores awarded by the ERC review panel during the second step of the ERC Synergy Grant review and the ranking range. The panel based its appraisal on prior individual reviews of the full proposal conducted by ERC panel members and external reviewers. The panel closely examined all the individual review reports and, while not necessarily subscribing to each and every opinion expressed, found that they provide a fair overall assessment. The comments of the individual reviewers were the basis for the panel's discussion and for the final recommendation of the panel, and are included in this report. The final conclusion of the panel is also included in this panel comment. The applicants are highly regarded specialists in their fields and they put forward a proposal with ambitious objectives that could potentially lead to breakthroughs in the planetary radar field. The synergistic collaboration among different disciplines allows to face the challenges of this proposal. However, the design methods and radar signal processing methods proposed to build such a radar are not clear. Also, the requested resources are not sufficiently justified and the description of the state of the art appears incomplete and not up-to-date. The panel was of the opinion that on a global level, this proposal was more incremental in nature than ground-breaking. The panel considered that the proposal is of overall high quality. However, based on the combined set of criteria used in the assessment it was not ranked highly enough to be retained for the final step of the evaluation. The panel therefore recommends that the proposal should not be retained for Step 3 and should not be considered for funding. 101118952 / 14.07.2023 Page 2 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 3, 14072023] REVIEWER COMMENTS The following individual reviews have been carried out independently prior to the panel meeting and do not necessarily reflect the panel's final opinion Reviewer 1 Research Project Ground-breaking nature and potential impact of the research project This proposal aims at consolidating past works of PIs for realizing incoherent scatter radars based on innovative measurement methods, signal processing and analysis that allows to achieve better performances. In particular, the project aims at demonstrating novel types of processes where microscopic interactions can drive a macroscopic abrupt phase transition. The project starts from theoretical work performed in the past by PIs and further used all around the world to develop radars. In order to do so, it addresses several paramount challenges related to mathematics, signal processing, statistics, as well as working with real antennas. Objectives of providing measurement instrumentation, algorithms and signal processing methodologies are quite ambitious and but it is not clear how much they are innovative and how much they are incremental with respect to the state of the art. Thus, while there is no doubt that this proposal has a high potential impact, its ground-breaking nature is not fully evident. Scientific Approach The project supports the develop new measurement data analysis and instrumentation for super radars and telescopes with optimized ubiquitous design backed by first-principles based mathematical inversion theory. The proposed scientific approach is definitely very feasible, as continuation of past activities of PIs, but not clearly high risk/high gain. The proposal as it combines the knowledge in mathematics, signal processing, statistics, and the capacity of all the individual principal investigators clearly goes far beyond their standalone research, and appears as a natural and necessary collaboration. In this sense, the combination of scientific elements indicated above is very important for addressing the scope and complexity of the radar discussed in this proposal, and the synergistic collaboration among different disciplines allows to face the challenges of this proposal. The proposal involve the development of methodologies for build methods and instrumentation for super radars and telescopes with optimized ubiquitous design, but even if the proposed research methodology and working arrangements are very appropriate to achieve the general goal of the project, their novelty is not sufficiently substantiated. Further, the timescale, the resources (including budget breakdown per PI) and the time commitment for each PI are adequate, even if the high involvement of the PI and senior staff of the corresponding organization are not sufficiently justified. Also, no justification is given for not having any PhD positions in the project. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Non-competitive To what extent do the PIs have the required scientific expertise and capacity to successfully Good execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Very good necessary to address the proposed research question? 101118952 / 14.07.2023 Page 3 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 4, 14072023] Comments (Optional for reviewers) The principal investigators demonstrate good competence in mathematics, signal processing, and statistics, which are required to successfully execute the project. However, their expertise and capacity in executing and conducting ground-breaking projects is not evident. 101118952 / 14.07.2023 Page 4 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 5, 14072023] Reviewer 2 Research Project Ground-breaking nature and potential impact of the research project The proposed research focuses on the definition and on the development of a rigorous theory of antenna design and signal processing to push the limits of radar for space science and astronomy. The objectives are ambitions and well beyond the state of the art. Research is high-risk high-gain and represents an extension of the work some of the PIs have done in the past. Scientific Approach The outlined scientific approach is very feasible given the expertise of the proposing team. The project objective go well beyond what a single PI can achieve and requires the expertise and the actual work of all the PI teams. The research methodology seems sound, but the time development of the project is not discussed in detail. The PIs will build on their previous expertise. The timescales seem justified, even if the 3.1 MEUR prototyping cost of PI2 is a large cost that is not fully justified. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Very good To what extent do the PIs have the required scientific expertise and capacity to successfully Excellent execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Very good necessary to address the proposed research question? Comments (Optional for reviewers) PIs 1 and 3 are well-known experts in the field of the project and have developed inelastic scattering radar methods that are presently applied and represent the state of the art. PI 2 has specific hardware expertise and a professional career not specifically research-based but important for the project. It is not clear from the proposal whether the project in the case of PI1 and PI3 will be carried on by a team or mainly by the PIs. 101118952 / 14.07.2023 Page 5 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 6, 14072023] Reviewer 3 Research Project Ground-breaking nature and potential impact of the research project Building the methodological capability for taking full advantage of large-scale radar and antenna systems, with the necessary mathematical underpinnings, is a highly important challenge with several applications, with the large-scale EISCAT et al a very nice case. The project aims at going clearly beyond state of the art in all its three objectives; there is high risk in both each of the fields, and in making them work well together. Scientific Approach This clearly could not be done by any of the PIs alone, and with this constellation of PIs seems feasible, given their collaboration history. The PIs have ample experience in their fields and the approaches are very well justified. Novel methodology will be developed to a great extent. The proposal is thinner on the practices of actually working togehter, and timescales. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Excellent To what extent do the PIs have the required scientific expertise and capacity to successfully Exceptional execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Excellent necessary to address the proposed research question? Comments (Optional for reviewers) No comments received 101118952 / 14.07.2023 Page 6 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 7, 14072023] Reviewer 4 Research Project Ground-breaking nature and potential impact of the research project The stated objective to generalize the team’s existing accomplishments in signal processing and analysis. The underlying importance of the scientific aims is not clearly developed. The team has a strong connection to the existing EISCAT_3D, which offers a promising facility for deployment of any new software developments, but no guarantee of their use. Some of the related goals have merit purely as theoretical developments (e.g., general principles for antennas), but these have limited scientific significance. The proposal asserts that this work will be state-of-the-art. The proposed work does not reach significantly across disciplines or bring together very different types of expertise. The basic combination of theory and experiment is not novel. Scientific Approach The PIs have made past progress in the field, which implies room for continued advancement. The work relies on a combination of theoretical and experimental/practical expertise, which no participant would fully cover alone. However, the text does not elaborate on the relationship between the specific theoretical and experimental contributions to the proposed work, and it does not demonstrate how the combined effort is greater than the sum of its parts. The program offers a common application in radar that could make use of the multiple developments proposed here. The work is computationally intensive, and expenses for computing facilities are included. Other significant hardware includes a test transmitter. The proposal reasonably includes significant costs for essential equipment. However, the apparent total lack of some basic versions or related equipment suggests that this team is not currently active in the field. The PIs will dedicate significant effort to this work. They will not involve any funded students in the program. Some funding is allocated to postdoctoral researchers, but their number, specific tasks or projects, and the duration of the employment are not described. The proposal does not give any timeline for the work, only the implication that the collection of projects will be completed over the lifetime of the funded program. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Non-competitive To what extent do the PIs have the required scientific expertise and capacity to successfully Good execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Non-competitive necessary to address the proposed research question? Comments (Optional for reviewers) The PIs do not all demonstrate the ability to do ground-breaking research. At least one brings primarily technical expertise. 101118952 / 14.07.2023 Page 7 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 8, 14072023] The proposal overall does not provide sufficient information to demonstrate that the full range of relevant and necessary skills will be available to complete this work successfully. 101118952 / 14.07.2023 Page 8 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 9, 14072023] Reviewer 5 Research Project Ground-breaking nature and potential impact of the research project This proposal includes three main challenges: 1) improved design of large-scale radars and radio-telescopes, 2) inverse problems, and 3) an optimized wideband antenna design, that are “oriented” towards: 1) the development of a methodology of the 2nd phase of EISCAT 3D, and 2) the design a distributed synthetic aperture at HF using smallsats. The topics are important, and ambitious. The smallsat synthetic aperture is mentioned, but then it vanished. Additionally, authors’ background in the field of smallsats is inexistent, or marginal, or at least this reviewer has not been able to find it. This topic could be by itself and ERC AdG. While in principle these overarching goals categorize as high risk-high gain, this reviewer finds some aspects of the proposal uneven: while the radar coding methods seem cutting-edge, there are other aspects listed below that are not or just incremental knowledge (antenna and array design, SDR back-ends etc.). It seems to this reviewer that most of the progress to be done will be conducted by one PI, and the other two, with complementary expertise will support the hardware and experimental activities. According to the PIs, current radar designs are sub-optimum, because the improper coding method, the non-optimal signal processing, and data analysis, the non-physical a priori models of the target, and the “wrong” way to estimate the errors in the measurements. This reviewer is a bit skeptical of some of the claimed gains: 1) Pure mathematical models: From a mathematical point of view a “an exact model of the target and the measurement” will be derived, but in my opinion it can be just that: a model, which can be significantly different from the reality. It is true that Physicists and Engineers have different views from Mathematicians, but it is experience and actual data, what helps to refine and improve the models, not just abstract mathematical concepts. 2) Antenna array geometries are a very mature area, and progress will likely produce only marginal improvements. Today, imaging algorithms have even been developed for synthetic aperture arrays with random and moving elements, such as in satellite swarms. 3) Tripolar and multipole antennas seem novel, and almost no public references are found. Although there are tripole and multipole antennas in the literature, they do not seem to refer the same type of antenna as the authors’, although having the same name. Scientific Approach The scientific approach is feasible, but the methodology could be better explained, at least in terms of the workflow, even without going into the typical WBS and Gantt. It is understood that it is research, and nobody knows what you are going to find, but you need to have a clear idea of the steps you want to follow, and what would be your plan B… not just which are the goals. The methodology seems a continuation of the one conducted in the original design of EISCAT 3D, that was not finished at that time, which is valuable. The proposal goes beyond what each individual PI can achieve, and the three profiles are complementary (mathematical background and hardware: SDRs, antennas etc.). However, this reviewer believes that neither the work is evenly distributed among the researchers, nor the topics where most of the cutting-edge research will be conducted are evenly distributed (the radar signal design). It is not the fraction of the time devoted to the project, it is about the topic of each one. The other two PIs, will bring their valuable knowledge, but their task seems more engineering design. Because of the above, and despite the efforts put by the authors in preparing this proposal, it seems to me more a juxtaposition of proposals, rather than their convolution. 101118952 / 14.07.2023 Page 9 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 10, 14072023] PIs’ commitment seems appropriate, also the project duration, although there is neither a detailed work package breakdown structure, nor a Gantt diagram detailing the project milestones and evolution. I think that engineering aspects are a bit undervalued. Even just developing a small array will bring numerous technical challenges, which can be -at least- as complex as the scientific challenges to be tackled. The requested resources are not properly justified. The statement “it is impossible to present a detailed cost breakdown structure at this point. We want to create and test new scientific/technological paradigms, which cannot be done by buying readily developed products from the market” is not acceptable to pretend to justify the lack of detail of expenses for up to ~4.5 M€. Total equipment requested -for example- is 3.5 M€, going mostly to PI 2. Frankly, it looks to me that there is n intention to fully equip a lab in the company. This is even more exaggerated when one revises budget item C1, with subcontracting costs of ~0.5 M€ for both PI 2 and PI 3, which are supposed to be used to develop this new instrumentation… so the detail of the 3.5 M€ to be spent in lab equipment must be clarified. On the other hand, consumables are very small (1500 € in total), despite we know that in a RF/electronics lab this budget item is orders of magnitude higher, especially with all the planned hardware developments. Publication fees for a 6-yer project are very low as well, assuming each article processing charges (APCs) equals 2 K€ per paper, this accounts for a total of 10 publication for PI 1, 3 for PI 2, and about 5 for IP 3, which is not a very high publication rate. Having said this, I do not mean that the quality of the work has to be measured by the “weight” (i.e. number of publications), instead of the quality of the individual publications, but for a ground-breaking project, which will supposedly provide many new results, I found this number a bit low. Last, but not least, it seems that only one postdoc will be hired by PI 2. Overall, despite subcontracting the development of new (TBD) instrumentation, the team seem a small one to tackle the magnitude of the proposal. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Exceptional To what extent do the PIs have the required scientific expertise and capacity to successfully Excellent execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Excellent necessary to address the proposed research question? Comments (Optional for reviewers) [Overall evaluation in first item is Exceptional because of PI 1 and PI 3, but in my opinion PI 2 lacks this ability] PI 1 has a moderate publication record (31 papers, 4131 citations, in Google Scholar), and has advised 15 PhD students. Most of the scientific developments of the project should be carried by him, in case it is granted. PI 1 benefit from an ERC AdG in 2010, and in the opinion of this reviewer he is the one with the largest demonstrated capability to conduct ground breaking research. PI 2 has a weak publication record (2 publications, 14 citations), but he has a long industry experience. He will be receiving most of the equipment, and subcontracting budget, and he will be in charge of the prototype developments. I wonder if they could be developed in house, or in collaboration with some university labs, such as Aalto University. PI 3 publication record is more difficult to find, it is not in Google Scholar, but in Scopus he appears to have ~60, with about ~1280 citations, and has advised 9 PhD students. In general, the 3 PIs bring complementary skills needed for the execution of a project as the one proposed. They are good, but I tend to disagree with their statement in Part B1, page 5: “All we together are world leading experts in the field. We have the best radar methods in the world, but ’selling’ them to practical engineers requires that we are impeccable in every respect and can describe our leadership in many different terminologies in mathematics, physics 101118952 / 14.07.2023 Page 10 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 11, 14072023] and engineering.” These 3 researchers are good, but there are great engineers and physicists out there with years-person of experience in radar design AND manufacturing. 101118952 / 14.07.2023 Page 11 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 12, 14072023] Reviewer 6 Research Project Ground-breaking nature and potential impact of the research project The scope of this proposal is to derive methods and instrumentation for new radars systems based mathematical inversion theory. The proposers claim that they want to advance every aspect of the design of this kind of radar system, from the antenna design to the solution of inverse problems, from the digital signal processor to the hardware architecture. The results of this project should represent the input (or an aid) of the second phase of EISCAT_3D. No doubt that the challenges addressed in this proposal are very important. However, if the general aims of this proposal are clear, how to achieve these ambitious goals is rather vaguely described in the proposal. The state of the art is largely incomplete. Reading the proposal it seems that the contribution of the radar community to the state of the art has not been significant after the publication of the 3rd volume of Larry Van Trees published more than 50 years ago. It seems that the proposers are not aware of many research works published in the most relevant radar and antenna design journals (IEEE Transactions of Aerospace and Electronic systems, IET Radar Sonar and Navigation, IEEE Transactions on Antennas and Propagation, IEEE Sensors Journal , IEEE Transactions on Microwave Theory and Techniques, IEEE Transactions on Geoscience and Remote Sensing , IET Microwaves Antennas and Propagation , IEEE Transactions on Signal Processing, to mention only a few of them). What are the present weaknesses and what are the improvements that the proposer want to achieve and how to achieve them is vary vaguely described. Hence, it is difficult to judge the risk that the research project does not entirely fulfill its aims, since these aims are not clearly stated. In the proposal, I find a description of EISCAT, but not a description of the state of the art in terms of antenna design, radar target modeling and radar signal processing methods for array radars. Scientific Approach The proposers certainly have the knowledge and experience to improve some aspects of EISCAT, e.g. the inversion methods adopted to retrieve the radar reflectivity map. Some of the stated objectives would require additional expertise in my opinion. The next-generation EISCAT radar (EISCAT_3D) will be a software-defined array radar. Hence, I would expect that the proposers clearly explain which new antenna design methods and radar signal processing methods to aim to derive, or how to improve existing methods, and how they want to measure the improvement. I cannot find this information in the current proposal. As an example, they mention the issue of mutual couplings, but they do not mention the current methods to take antenna coupling into account and how to improve with respect to the state of the art. In the proposal I find the list of objectives very vaguely stated, but not the milestones to achieve the final goals. I cannot find the organization of the various temporal phases of the work. The methodology is also very vaguely described and so I doubt it is appropriate to achieve the ambitious goals of the project. In the description of the objectives, I often find the word “optimal” but it is never stated what is the criterion of optimality adopted here. For example it is not clear what is the definition of optimal array geometry, the same for the optimal radar code. The goodness of a radar code can be measured by using different criteria. It is not clear how they can design a radar code with no side lobes. In equation 1 they cite space-time Gaussian white noise, but in radar scenarios the clutter is always present, often correlated and sometimes non Gaussian. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Excellent To what extent do the PIs have the required scientific expertise and capacity to successfully Very good execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Very good necessary to address the proposed research question? Comments (Optional for reviewers) The PI is a mathematician expert in solving EM inverse problems and its applications. He obtained the ERC’s 101118952 / 14.07.2023 Page 12 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 13, 14072023] Advanced Grant in 2010. PI2 has a vast experience in wireless systems design with applications to satellite communications, with years of professional experience. PI3 is also an expert in incoherent scatter radar measurement techniques, ionospheric radar measurements, inverse problems and in particular Bayesian inverse methods. None of them are expert of radar signal processing. In the list of their works I do not find anyone published in the most relevant radar and antenna design journals, such as e.g. IEEE Transactions of Aerospace and Electronic systems, IET Radar Sonar and Navigation, IEEE Transactions on Antennas and Propagation, IEEE Sensors Journal , IEEE Transactions on Microwave Theory and Techniques, IEEE Transactions on Geoscience and Remote Sensing , IET Microwaves Antennas and Propagation , IEEE Transactions on Signal Processing, to mention only a few of them. 101118952 / 14.07.2023 Page 13 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 14, 14072023] Reviewer 7 Research Project Ground-breaking nature and potential impact of the research project The proposed research certainly addresses important challenges, from both the theoretical and the engineering point of views. The proposed large-scale radio array is not new as idea, since there are already at least 3 operational systems in the world: SKA, EISCAT and Sanya. What is new is the theoretical rigorous approach, based on the mathematical inversion theory, to build optimized antennas and a planetary radar. These theoretical results seem to be already available to the applicants (at least partially), but never applied in a real system as could be EISCAT_3D. Difficult to judge the risk of the project. There are not many details and the applicants are so sure of their success that everything seems to be possible. It needs only to be applied, beating the conservative attitude of EISCAT. Anyway, the gain would be high, particularly in terms of image resolution or of data processing time, that nowadays is too long. Also, the optimization and testing of antennas with a radiation pattern frequency independent over a wide range of frequencies can have a large impact on other related areas. The project needs expertise from different braches of science and engineering. In that sense, it is interdisciplinary Scientific Approach The proposal is not always clear and detailed. Sometimes some technique is cited, but without reference (see for instance Turowski brushes). The proposal talked about “perfect radar code” without sidelobes, that sounds quite strange to me. At least, it is not clear what they mean with “sidelobes” here. It would have been beneficial to explain what is the peculiarity of these codes that make them "perfect" for this application. Pulse compression and compression codes are standard in many radar applications. The idea of the tripolar antennas, already partially experimented by HI2, is interesting. The applicants mention also “new kinds of beamforming signal processing”, necessary for their system, but never provide any hint on these algorithms. Moreover, at the beginning of the section “Methodology”, the applicants waste time and space in criticizing the common engineering approach and common lack of knowledge of what they think is basic, instead of really describing why their approach will work much better. The other interesting idea is to use a fleet of small satellites for measuring the HF radio waves that generally do not penetrate the ionosphere. A radio map of the universe at low frequency would be a great achievement. The “Mathematical proofs to replace preliminary results on antenna geometry” is quite generic. Not clear what has been done and how difficult will be the remaining part. Conversely, the section on antenna prototyping is better written and shows that the applicants have a very good knowledge of this practical aspects (not only about theory). The 3 research groups integrate perfectly their expertise, covering the aspects of EM modeling, inverse problems, and mathematical theoretical analysis. PIs commitment is satisfactory but I’m surprised that there are not student salaries nor post-docs for the corresponding PI and 3rd PI. Moreover the travel expenses seem to be a bit high. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Excellent To what extent do the PIs have the required scientific expertise and capacity to successfully Excellent execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Very good necessary to address the proposed research question? Comments (Optional for reviewers) My impression is that they overstate a bit their competence. I looked at google scholar and I have not found many 101118952 / 14.07.2023 Page 14 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 15, 14072023] papers and citations. Anyway, they seem to have the right expertise and they complement and integrate each other in this project. I suggest that they hire also PhD students and post-docs. 101118952 / 14.07.2023 Page 15 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 16, 14072023] Reviewer 8 Research Project Ground-breaking nature and potential impact of the research project 1. To what extent does the proposed research address important challenges? The proposal aims to develop advanced methods and instrumentation for super radars and telescopes, incorporating optimized design principles supported by rigorous mathematical inversion theory. The proposed research will tackle a range of significant challenges in the fields of mathematics and engineering. These challenges encompass large-scale radars and telescopes, inverse problems, as well as antenna, antenna array, and digital signal processing engineering. 2. To what extend are the objectives ambitious and beyond the state of the art (e.g. novel concepts and approaches or development between or across disciplines)? The proposal outlines a set of ambitious objectives that seek to address fundamental mathematical questions in antenna theory, enhance optimization results for antenna arrays, develop prototypes of advanced antenna elements and arrays, and establish a meticulously optimized planetary radar system. The overarching goal is to create opportunities for groundbreaking advancements in radar and antenna technology, space research, and Bayesian inversion theory, thereby paving the way for new avenues of exploration and potential breakthroughs. 3. To what extend is the proposed research high risk-high gain (i.e. if successful the payoffs will be very significant, but there is a high risk that the research project does not entirely fulfill its aims)? The proposed research carries a high risk-high gain profile. If successful, it has the potential to provide groundbreaking insights into radar and antenna technology, as well as advance the field of mathematical inversion theory by addressing significant mathematical and engineering questions. As mentioned, the proposal presents a series of ambitious objectives that, if achieved, could lead to substantial advancements. The three Principal Investigators (PIs) bring together their synergistic and complementary expertise in their respective fields to tackle these objectives. However, it is important to note that the proposed research activities may require a substantial research team to be successfully executed. Scientific Approach 1. To what extent is the outlined scientific approach feasible bearing in mind the extent that the proposed research is high risk-high gain? The proposed research activities align with the expertise of the Principal Investigators, who are recognized as leading experts in their respective fields with well-established research records. Within the proposal, the Principal Investigators have presented preliminary results, demonstrating progress in the research. Additionally, the outlined scientific approaches are deemed feasible. 2. To what extent does the proposal go beyond what the individual Principal Investigators could achieve alone? This proposal is interdisciplinary in nature, necessitating expertise in mathematics, as well as radar and antenna theory and technology. The attainment of the proposed objectives is unlikely to be accomplished by any individual Principal Investigator working alone. 3. To what extent do the Principal Investigators succeed in proposing a combination of scientific approaches that are crucial to address the scope and complexity of the research question to be tackled? This proposal demonstrates excellent writing, effectively presenting the state-of-the-art and outlining the objectives. The proposal provides a clear description of the challenges associated with each aspect and outlines the scientific approaches that will be employed to address the research questions. 4. To what extent are the proposed research methodology and working arrangements appropriate to achieve the goals of the project? The proposal outlines three objectives, where the Principal Investigators describe the research methodology and the actions they will undertake. However, further clarification is needed regarding the specific implementation details of their proposed methods. To strengthen the proposal, providing additional details about the proposed methods would enhance the understanding of how the Principal Investigators plan to accomplish their objectives. 5. To what extent does the proposal involve the development of novel methodology? 101118952 / 14.07.2023 Page 16 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 17, 14072023] The Principal Investigators aim to advance Bayesian inversion theory and enhance antenna theory and technology. However, it remains unclear how the proposed methodology will surpass the existing results in these areas. While the intentions of the Principal Investigators to develop Bayesian inversion theory and improve antenna theory and technology are clear, further clarification is needed to demonstrate how the proposed methodology will go beyond the current state of research. 6. To what extent are the proposed timescales, resources and PI commitment adequate and properly justified? The commitment of the Principal Investigators (PIs) is sufficient and appropriately justified. However, it is unclear what additional personnel would be involved in the proposed research, aside from the three PIs. Given the ambitious objectives of the proposal, it is recommended to include junior researchers to complement the expertise of the Principal Investigators and support the successful execution of the research. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Exceptional To what extent do the PIs have the required scientific expertise and capacity to successfully Excellent execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Excellent necessary to address the proposed research question? Comments (Optional for reviewers) No comments received 101118952 / 14.07.2023 Page 17 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 18, 14072023] Reviewer 9 Research Project Ground-breaking nature and potential impact of the research project The proposed research furthers the state of the art in imaging of phenomena observed in radio wavelengths. The current approaches are compute intensive, and progress is required on many fronts such as proposed here. This is an important challenge to undertake, relevant to many fields. The objectives are truly ambitious, being a dramatic extension and subsequent demonstration of previous work. The activities are geared to facilitating practical high performance designs in a number of disciplines. The overall programme of work is high risk, purely because of its scope (advancing theory, instrumentation and applications in the same programme. Should it succeed, it is very high value. Scientific Approach The scientific approach has been proven feasible, and the work is geared to demonstrating practicality in implementing high performance systems fulfilling the promise. There is a distinct mix of theoretical, experimental and engineering expertise in the team which is not present in the same proportion in any one individual. The PIs do indeed succeed in proposing credible approaches to the ongoing development and implementation of their ideas. The work is multifarious and scientific and engineering terms and the interdependences are not clear to achieving the ultimate outputs of designs of advanced instruments and systems. The working arrangements are nonetheless suited to the work. Novel methodology is the main thrust of the proposal. The proposal is not clear on programmatics; as noted above, the dependencies are not drawn out, and the task breakdown not fully clear. In envelope terms, the duration and costs are as expected for a programme that produces designs and prototypes. It is not clear how much access to EISCAT and Sanya facilities are required to achieve the goals. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Excellent To what extent do the PIs have the required scientific expertise and capacity to successfully Exceptional execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Excellent necessary to address the proposed research question? Comments (Optional for reviewers) The track records and decades of experience of the PIs in this domain are a testament to their future performance. 101118952 / 14.07.2023 Page 18 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 19, 14072023] Reviewer 10 Research Project Ground-breaking nature and potential impact of the research project The proposal is rather vague to my taste. As main research question, I found: How to construct rigorous antennas, arrays, and telescopes to reliably answer fundamental questions in space sciences and astronomy? Obviously that is a very broad ambition. The main target then is to develop EISCAT_3D to a world leading (european) ionospheric scintillation radar using the newest methods and hardware technology. It is mentioned that a similar Chinese instrument is operational but that the proposers can do "better" and improve on their own earlier work. Indeed, as main challenge the authors mention creating an "optimised ubiquitous design backed by first-principles based on mathematical inversion theory." This can mean anything, from very hard to very easy. It is not possible to judge if this is high-risk high-gain, and what the ambition level is. The "state-of-the-art" section in the proposal does not describe the state of the art in any way. The newest reference in this section is from 2015. Both in Part B.1 and B.2, the section discusses topics around the EISCAT radar, inversion, and antenna/array design, but rather than providing context, it (too) often just refers to the PIs work from decades ago - is that the state-of-art? From this section, I do not learn about ionosphere scintillation (what is its scientific purpose, which instruments are available/state-of-art, what significant improvement is needed, why is that a fundamental scientific challenge). The rest of the proposal also doesn't place itself in broader perspective. I am not familiar with ionospheric scintillation, but I know that "inversion" is a major and active research topic in mathematics and engineering, with many branches. Nothing is mentioned regarding current viewpoints and approaches, or even the specific branch that is taken here, and any specific unique idea that the PIs will pursue and that will shaken up this field. Current research on computational imaging is entirely going into the direction of machine learning, loop unrolling, and neural network implementations; I would expect at least a reference to this. Scientific Approach The proposal does not describe in any detail what questions will actually be addressed. Three objectives are phrased as reseach topics only. Then in B.2 seven 'questions' to be addressed are given, but these contain 4 demonstrators, and the other three say "improve", "open new avenues", "answer the fundamental questions": these are all vague and not concrete questions. The objectives lack focus. I understood that the main objective is to propose (and demonstrate) alternative designs to EISCAT_3D. Although PI2 was involved in EISCAT until 2014, the current relation to EISCAT_3D remains vague: is this project aligned at all (and how) to the current instrument, whose commissioning is said to be stalled? It appears to me that there is no alignment, or even contact, to the current EISCAT_3D team, if even such a team exists (no mentioning of an institute, steering board, or otherwise). Among the objectives is to design a satellite system for HF observations from space; described in half a page. This by itself is a major effort. No reference is given to existing or past international efforts in this direction, such as OLFAR, NOIRE (each with large teams and running over a decade), SULFRO, etc. It appears that this objective is "off-the-cuff" and not substantiated in any way. It is also unclear what this objective has to do with the main objective (design studies for EISCAT_3D). I do not see any satellite expertise within the project. The description of another objective: "design study for a European planetary radar", is at the same superficial level. The proposal lacks any division of the proposed objectives into practical activities (such as work packages) and clear outcomes (such as milestones), and time division within the proposed 72 month duration. It is impossible to judge how much manpower each objective will require, and what exactly will be realized in the end. In short, this proposal is not a project definition at all. Principal Investigators 101118952 / 14.07.2023 Page 19 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 20, 14072023] To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Good To what extent do the PIs have the required scientific expertise and capacity to successfully Good execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Non-competitive necessary to address the proposed research question? Comments (Optional for reviewers) The cPI and PI2 are essentially retired, although still active. I believe that their advanced age constitutes a risk to the execution of the project. The proposal has no information on the size of the team that will be assembled to carry out the work. Only the proposed budget indicates that a few other (senior) persons will be involved, and 1 PD, but no PhD students. The role and tasks of these people are not described. Nothing is said about the embedding of the project in the host institutes (access to students or other resources?) In my opinion, the manpower allocated to the project is far insufficient to carry out the ambitious goals. The CVs of cPI and PI2 are good, but hardly anything is mentioned that relates to the past 10 years. PI1 represents a small but very relevant engineering company; the size of the company is not disclosed ("it represents 100 man-years of expertise"). If this project is carried out, it will be the main activity of the company. 101118952 / 14.07.2023 Page 20 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 21, 14072023] Reviewer 11 Research Project Ground-breaking nature and potential impact of the research project The investigators propose to improve the capabilities of EISCAT by improving the analysis methods and adding an outlier array to EISCAT-3D. The analysis methods have have been developed by the investigators close to 10 years ago and have been adopted by the Chinese Sanya radar. Using those methods in EISCAT-3D therefore feels like engineering work instead of ground-breaking scientific work to me. This feeling is reinforced by the fact that the investigators indicate that they want to provide thorough mathematical proof for results that are already established by numerical experimentation. Although adding an outlier array to EISCAT-3D will further enhance the capabilities of the EISCAT radar, the focus of this project is on optimisation of the antenna design, not on the science to be done with EISCAT exploiting those enhanced capabilities. Would it have been more cost-effective to simply realise an outlier array using an already proven array design? The investigators note that the inversion methods used for incoherent scatter radars may be of interest to other fields, such a radio astronomy. Unfortunately, based on the lack of references to relevant work, the investigators show no sign of being aware of related efforts in the radio astronomical domain on the use of Bayesian Inference and Information Field Theory for image reconstruction and uncertainty quantification. I am thus unable to conclude that this project will build on top of and advance the current state-of-the-art in that area. Scientific Approach As demonstrated by the adoption of the proposed analysis methods by the Sanya radar, adoption of those methods by the EISCAT-3D radar has the potential to make a big step forward in instrument performance. This is high-gain, but not high-risk as it concerns an engineering / development effort to realise. To realise the ambition of improving EISCAT-3D performance, the complementary expertise of the investigators is required. This is underscored by the fact that the investigators have already collaborated fruitfully in the past. This means that this project is not a new collaboration, but will benefit from earlier collaborative endeavours. The project contains several exploratory avenues that do not (directly) contribute to the main goal of improving EISCAT-3D performance. One such avenue is to use the mathematical model to optimise the array geometry for a radio-astronomical space-based HF array. Such unrelated explorations dilute the focus of the project. Moreover, the investigators do not make reference to several ongoing efforts around the globe towards space-based HF arrays for radio astronomy, which at least indicates a poor connection of the investigators with such projects, reducing the chances of those projects benefitting from this work. Another, in my opinion unneeded, exploratory avenue is antenna optimisation for an EISCAT-3D outlier array. Wouldn't it be more cost-effective to simply realise this using proven antenna design? My concerns regarding the cost-effectiveness of the proposal grew further due to some comments supposed to substantiate the budget, such as "We will produce the best we can with the 4 MEuro 'extra', but what 'best' is will only become clear as the project is proceeding". This does not give me confidence that the project has been thoroughly thought through. Principal Investigators To what extent have the PIs demonstrated the ability to conduct ground-breaking research? Excellent To what extent do the PIs have the required scientific expertise and capacity to successfully Excellent execute the project? To what extent does the Synergy Grant Group successfully demonstrate in the proposal that it brings together the know-how - such as skills, experience, expertise, disciplines, teams - Excellent necessary to address the proposed research question? 101118952 / 14.07.2023 Page 21 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Associated with document Ref. Ares(2023)4912962 - 14/07/2023 [Panel: SyG2D, Page 22, 14072023] Comments (Optional for reviewers) The investigators have some long-standing collaborations, which gives confidence for further collaboration. I found it amusing to note that the ten-year track record of the corresponding PI mainly lists achievements of (well) over 10 years ago In the ten-year track record of PI3, it is stated that "the [EISCAT] organisation is reluctant to replace the methods with our new far better ones." The way this is formulated gave me a sense of frustration of the PI with the EISCAT organisation, which may affect a fruitful collaboration with the EISCAT organisation needed to get the proposed methods integrated in the EISCAT radar. Another sign of this is that, for some reason or another, the EISCAT organisation did not pick up these methods close to 10 years after them being proposed while the developers of the Sanya radar did. When reading carefully, I sense this frustration thoughout the proposal, which makes me concerned about the impact this group can make within the EISCAT community. 101118952 / 14.07.2023 Page 22 of 22 European Research Council Executive Agency, Scientific Management Department, Place Rogier 16, BE-1210 Brussels http://erc.europa.eu  Digitally sealed by the European Commission Date: 2023.07.14 13:22:30 CEST This electronic receipt is a digitally signed version of the document submitted by your organisation. Both the content of the document and a set of metadata have been digitally sealed. This digital signature mechanism, using a public-private key pair mechanism, uniquely binds this eReceipt to the modules of the Funding & Tenders Portal of the European Commission, to the transaction for which it was generated and ensures its full integrity. Therefore a complete digitally signed trail of the transaction is available both for your organisation and for the issuer of the eReceipt. Any attempt to modify the content will lead to a break of the integrity of the electronic signature, which can be verified at any time by clicking on the eReceipt validation symbol. More info about eReceipts can be found in the FAQ page of the Funding & Tenders Portal. (https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/support/faq) Commission européenne/Europese Commissie, 1049 Bruxelles/Brussel, BELGIQUE/BELGIË - Tel. +32 22991111