Enhancing the Security, Privacy and Robustness of AI Models and Systems (SecureAI)

 

SCOPE & EXPECTED OUTCOMES :

 

This topic aims to strengthen the resilience of AI systems and algorithms against various threats and attacks, such as enhancing their resilience against adversarial attacks, backdoor injections, and data poisoning.

Proposals are expected to contribute to one or more of the following:

• Robust AI models and systems capable of resisting different classes of adversarial manipulation;
• Innovative defence mechanisms for AI models and systems against new attack families;
• Methodologies for detecting and mitigating attacks, such as data poisoning, backdoor exploitation and misclassification;
• AI systems leveraging privacy-enhancing technologies that maintain data confidentiality and regulatory compliance, enabling trusted in-house AI deployments (e.g., for governments and enterprises).

Proposals should develop real-time anomaly detection, mitigation techniques to defend against adversarial attacks and robust federated learning techniques, in synergies with leading efforts on AI transparency, and in compliance with the AI Act.

The topic is expected to:

• Develop robust AI models resistant to adversarial attacks.
• Improve detection of manipulated or poisoned training data. Advancing methodologies to identify and mitigate compromised datasets, leveraging techniques such as anomaly detection, provenance tracking, and automated data validation mechanisms.
• Address the concept of Private AI by developing mechanisms that enable AI models to be trained, deployed and operated in privacy-preserving environments, particularly for sensitive use cases, as for example for government and enterprise settings.

 

The consortium should gather at least 3 independent legal entities. These partners must be from 3 different countries (EU or Associated), with at least one from an EU Member State.

Must be established in an EU Member State or Associated Country, or a third country as explicitly eligible in Horizon Europe.

Any legal entity with operational and financial capacity can participate: Universities, research centres, hospitals, SMEs and start-ups, NGOs, patient associations, public bodies, SSH experts, …