The CSIC offers scholarships for the introduction to a research career in the context of the JAE Intro Programme. The IIIA offers seven projects to be carried out at our research institute:

Projects and mentors:

1. Estudios de la ciencia y la tecnología aplicados a la robótica social asistencial (JAEINT24_EX_0111):
   Mentor: Núria Vallès Peris
   La presente oferta tiene como objetivo general ofrecer a la persona candidata una formación teórica y empírica en los Science and Technology Studies (STS) aplicada a la robótica social. Los STS son una perspectiva interdisciplinar que integra la aproximación de las ciencias sociales y las humanidades en el estudio de la ciencia y la tecnología. Específicamente, la persona seleccionada participará en proyectos sobre el estudio de las controversias éticas, políticas y sociales alrededor de la robótica social asistencial. Desde hace un par de décadas los robots sociales se están desarrollando para realizar tareas de cuidados en diversos ámbitos (procesos terapéuticos, acompañamiento a la soledad, apoyo en tareas cotidianas, entre otros). Aparte de los retos técnicos que supone el desarrollo de estos artefactos, por su capacidad de interpelar nuestra propia humanidad, la robótica social también plantea retos socio-políticos relevantes y plantea una serie de debates alrededor de cuestiones como el papel de los cuidados en las sociedades contemporáneas, el modelo de gestión de los sistemas públicos de salud o la participación de colectivos concernidos de pacientes y profesionales sanitarios/asistenciales en el diseño tecnológico. Mediante su participación en proyectos de investigación con robots sociales se ofrecerá a la persona seleccionada una formación teórica sólida en el ámbito de los STS aplicados a la robótica, así como también una formación sobre metodologías de análisis cualitativas (entrevistas en profundidad semi-estructuradas y etnografía focalizada). Esta formación podrá servirle a la persona candidata como parte de su trabajo fin de Grado (TFG) o trabajo fin de Máster (TFM) en áreas de las ciencias sociales o humanidades (sociología, antropología, psicología social, historia, etc.), con un claro interés en el estudio social de la inteligencia artificial y la robótica. Por otro lado, se pretende fomentar el interés en la realización de una tesis doctoral mediante la preparación conjunta de una propuesta FPU u otras convocatorias afines.
    
2. Explainable AI by Way of Embodied Cognition (JAEINT24_EX_0309)
   Mentor: Marco Schorlemmer
   Machine-learning systems based on deep neural networks are currently pattern-matching black boxes that make it difficult for both developers and users to understand when a particular set-up of a neural network is going to be successfully trained and deployed in a trustworthy and robust manner. This project aims to make deep-learning architectures more transparent to developers and users alike by increasing their degree of explainability by design, with those built-in concepts that are currently lacking and which may help to reveal their underlying assumptions and behaviour. We will draw from the insights of contemporary cognitive science on embodied cognition, which claims that human conceptualisation and understanding are largely grounded on our bodily experience and the interactions we establish with the environment at a sensorimotor level. We will explore how, by taking this perspective of cognition as a reference, we can contribute to one of the fundamental ethical objectives of AI for the coming years, namely the objective of explainability. At IIIA-CSIC we have developed mathematical and computational models of embodied cognition, applying them to mathematical conceptualisation, diagrammatic reasoning and musical creativity. For this particular project, we will team up with researchers from UAB’s Philosophy Department with expertise in embodied and enactive approaches to cognition. This is a highly interdisciplinary project, bringing together techniques from cognitive linguistics, computer science, mathematics, and philosophy.
    
3. Automated design of ethical learning environments for autonomous cars with multi-objective and deep reinforcement learning (JAEINT24_EX_0436)
   Mentor: Juan A. Rodríguez-Aguilar
   Reinforcement learning (RL) is the most prominent framework for sequential decision-making (with or without uncertainty) nowadays. Since the surge of Deep Reinforcement Learning (DRL), there has been an explosion of algorithms to solve sequential problems that range from beating world champions of chess and Go to winning at realistic racing simulators like Gran Turismo. An RL agent learns its behaviour via a trial-and-error scheme: while learning, the agent keeps acting upon its environment, and after each action, it receives a reward as feedback, and also observes how its action changes the environment. By repeating this loop, the agent eventually learns the sequence of actions that maximises its accumulation of rewards. A fundamental problem in RL is how to design learning environments. The research question is how to automate the design of an environment for a learning agent so that the behaviour that it learns is guaranteed to be optimal. Our group has pioneered the optimal design of “ethical” environments, namely learning environments where the optimal policy that an RL agent learns is guaranteed to be ethical [1,2]. Unfortunately, such techniques do not scale when considering large, actual-world environments. The goal of this scholarship will be to explore the design of novel techniques for the design of ethical environments for autonomous cars. We envision three main challenges: (1) the adaptation of an autonomous driving simulator to consider ethical actions; (2) the use of Multi-objective Reinforcement Learning techniques to develop an “approximate”, extended version of the technique developed in [1,2]; and (3) the empirical evaluation of the ethical policies learned by autonomous cars with DRL. From a research perspective, this project will depart from multi-agent cooperative learning to focus on “mixed” learning environments. These are considered to be more challenging and have been much less explored in the literature.
   
   [1] Rodriguez- Soto M, Lopez-Sanchez M, Rodriguez Aguilar JA (2021) Multi-objective reinforcement learning for designing ethical environments. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp 545–551.
   [2] Rodriguez-Soto, M., Lopez-Sanchez, M. & Rodriguez-Aguilar, J.A. Multi-objective reinforcement learning for designing ethical multi-agent environments. Neural Computing & Applications (2023). https://doi.org/10.1007/s00521-023-08

4. Machine Learning for Prediction of Epilepsy Crisis from Electroencephalograms (JAEINT24_EX_0607)
   Mentor: Jesús Cerquides
   This research project aims to employ machine learning methodologies to predict epilepsy crises by analyzing publicly available electroencephalogram (EEG) data. Recognizing the challenges associated with timely prediction of epilepsy seizures, the study focuses on utilizing existing datasets to develop a predictive model. By leveraging machine learning algorithms, particularly deep learning models, the research will involve preprocessing publicly accessible EEG data, extracting relevant features, and implementing advanced classification techniques. The objective is to create a reliable and accurate predictive tool capable of discerning patterns and subtle changes in EEG signals preceding epileptic seizures.

5. New Variants of the MiCRO Negotiation Strategy (JAEINT24_EX_0826)
   Mentor: Dave de Jonge

   BACKGROUND: The topic of automated negotiation deals with the question how autonomous software agents can negotiate with each other. Specifically, it deals with scenarios in which two or more agents need to solve a problem together, even though they have conflicting interests. This means that the agents need to compromise and find a solution that is acceptable to everyone. In order to come to an agreement, the agents may propose solutions to each another, and each agent may accept or reject the proposals it receives from the other agents. A typical example is the case of a buyer and a seller that are bargaining over the price of a car. While the seller aims to sell the car for the highest possible price, he still needs to make sure the price is low enough for the buyer to accept the deal. Recently, an extremely simple new negotiation algorithm, called MiCRO, was introduced by Dr. Dave de Jonge which was shown to outperform almost all existing state-of-the-art negotiation algorithms, even though MiCRO is much simpler than those other algorithms. Unfortunately, however, MiCRO is only applicable to negotiations between no more than two agents, and only to problems for which the number of possible solutions is relatively small (less than a million). To deal with these limitations, dr. de Jonge has proposed some ideas on how MiCRO could be generalized to negotiations among more than two agents, and to negotiations with a larger number of possible solutions (several millions). GOALS OF THIS PROJECT: The goal of this project is for the student to implement these ideas (in Java or Python), perform experiments, and determine how well these new variants of MiCRO perform against state-of-the-art negotiation algorithms, and under which parameter settings. And perhaps, based on the results of those experiments, the student could even figure out ways to improve MiCRO even further. Optionally, the task can be made more challenging, by trying to implement an even more advanced algorithm that is applicable to astronomically large test cases (e.g. with 10 to the power 100 possible solutions). This would require the use of more complex search techniques, such as genetic algorithms or tree search.

6. Advanced AI for Immersive Training Simulations (JAEINT24_EX_1586)
   Mentor: Jordi Sabater-Mir
   The project focuses on the development and implementation of realistic Non-Player Characters (NPCs) within simulated environments for training purposes. The objective is to enhance the immersive quality of training simulations by populating them with NPCs that exhibit lifelike behaviours and responses. The work involves designing and programming NPCs with advanced artificial intelligence algorithms including Large Language Models (LLMs) and other AI technologies, enabling them to adapt dynamically to changing scenarios, interact convincingly with trainees, and simulate a wide range of human-like behaviours. The goal is to push the boundaries of immersive training simulations, providing trainees with more realistic and challenging scenarios that better prepare them for real-world situations.

7. Neurosymbolic AI: from Theory to Applications (JAEINT24_EX_1596)
   Mentor: Vicent Costa
   Neurosymbolic artificial intelligence is a recent domain in artificial intelligence (AI) that seeks to merge the knowledge-based symbolic approach with neural network-based methods. It is mainly motivated by application-level regards (e.g., explainability and interpretability) and algorithmic-level considerations (e.g., long-term planning and analogy) and intends to merge the strengths of both approaches and overcome their corresponding drawbacks. The main goal of this project is to integrate principles and aspects from both approaches and to design hybrid systems in this emerging field of AI. The application domains would be related to tutors' previous works, especially to issues concerning people with different kinds of disability (e.g., evaluation of the quality of life of people with mental distress). The ideal candidates for this fellowship have excellent programming skills and knowledge of logic and theoretical computer science and are concerned with the ethical aspects of AI systems design.