Pause
Lecture
Moteur de recherche d'offres d'emploi CEA

Decentralised federated learning in a dynamic environment H/F


Détail de l'offre

Informations générales

Entité de rattachement

Le CEA est un acteur majeur de la recherche, au service des citoyens, de l'économie et de l'Etat.

Il apporte des solutions concrètes à leurs besoins dans quatre domaines principaux : transition énergétique, transition numérique, technologies pour la médecine du futur, défense et sécurité sur un socle de recherche fondamentale. Le CEA s'engage depuis plus de 75 ans au service de la souveraineté scientifique, technologique et industrielle de la France et de l'Europe pour un présent et un avenir mieux maîtrisés et plus sûrs.

Implanté au cœur des territoires équipés de très grandes infrastructures de recherche, le CEA dispose d'un large éventail de partenaires académiques et industriels en France, en Europe et à l'international.

Les 20 000 collaboratrices et collaborateurs du CEA partagent trois valeurs fondamentales :

• La conscience des responsabilités
• La coopération
• La curiosité
  

Référence

2023-29862  

Description de l'unité

Le Laboratoire Instrumentation Intelligente, Distribuée et Embarquée (LIIDE) a pour mission de développer une plateforme mixte, matérielle et logicielle, pour concevoir les fonctionnalités de l'instrumentation du futur. Le laboratoire développe conjointement 1) le volet matériel, visant des cartes électroniques polyvalentes et modulaires, accompagnées des logiciels nécessaires à leur fonctionnement, pour couvrir une large gamme de technologie de capteurs ; et 2) des fonctionnalités innovantes d'intelligence artificielle pour la mesure répartie et l'apprentissage frugal et distribué.

Le laboratoire est ancré dans un environnement riche centré autour de l'instrumentation numérique pour le contrôle, le monitoring et le diagnostic. Le département auquel il appartient s'appuie sur une large gamme de capteurs (fibres optiques, capteurs piézo-électriques, sondes Courants de Foucault, rayons X) ainsi que sur des plateformes d'expérimentation de pointe. Les applications sont principalement focalisées sur le contrôle non-destructif (Non-Destructive Evaluation - NDE) ou la surveillance de l'état de santé de structures (Structural Health Monitoring - SHM).

Description du poste

Domaine

Mathématiques, information  scientifique, logiciel

Contrat

Stage

Intitulé de l'offre

Decentralised federated learning in a dynamic environment H/F

Sujet de stage

Federated learning was introduced in 2016 by Google [1] as a new machine learning paradigm where multiple entities (clients) collaborate in solving a machine learning problem under the coordination of a central server. Each client trains a local model using its private data, and only model parameters are exchanged between the clients and the server, without exposing clients' private data [2]. However, the original architecture of federated learning (centralised federated learning) is highly dependent on a central server for orchestrating the training process. The objective of this internship is to investigate the solutions for enabling decentralised learning without the central server, taking into account the challenges including statistical heterogeneity and dynamics of the learning environment.

Durée du contrat (en mois)

6

Description de l'offre

The most prevalent architecture of federated learning (i.e., centralised federated learning) follows a client-server architecture where a central server orchestrates the entire federated training process from sending training tasks to clients to receiving clients’ model updates and aggregating them. The dependence on the central server subjects this architecture to problems such as single-point of failure, man-in-the-middle attacks, as well as trust and fairness issues [3-4].

 

Decentralised federated learning (DFL) has emerged in 2018 with the aim of distributing the aggregation of model parameters between clients [5], thereby removing the dependency on a single server and its associated risks. Though a promising alternative to centralised federated learning, DFL requires much consideration on various fundamental aspects such as architectures (fully decentralised or semi-decentralised), network topologies (mesh, cluster, star), aggregation distribution and algorithms, and communication schemes (synchronous or asynchronous) [5]. In realistic applications, local data of different clients participating in federated learning are commonly heterogeneous (i.e., non-IID). In addition, system dynamics (e.g., client disconnection, message losses) is a common concern in such a distributed setting. The focus of this internship is to study and analyse DFL; we seek to understand the impacts of these challenges on the design of DFL.

 

The internship will proceed as follows:

  • Conduct a literature review on DFL to identify the fundamental considerations for DFL (e.g., architectures, network topologies, optimisation and aggregation algorithms, communication mechanisms);

  • Identify the impacts of statistical heterogeneity and system dynamics on DFL;

  • Study the state-of-the-art solutions to mitigate the identified impacts;

  • Conduct an empirical evaluation of the solutions.

 

 

[1] McMahan, B., Moore, E., Ramage, D., Hampson, S. and y Arcas, B.A., 2017, April. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics (pp. 1273-1282). PMLR.

[2] Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A.N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R. and D’Oliveira, R.G., 2021. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2), pp.1-210.

[3] Gabrielli, E., Pica, G. and Tolomei, G., 2023. A survey on decentralized federated learning. arXiv preprint arXiv:2308.04604.

[4] Yuan, L., Sun, L., Yu, P.S. and Wang, Z., 2023. Decentralized Federated Learning: A Survey and Perspective. arXiv preprint arXiv:2306.01603.

[5] Beltrán, E.T.M., Pérez, M.Q., Sánchez, P.M.S., Bernal, S.L., Bovet, G., Pérez, M.G., Pérez, G.M. and Celdrán, A.H., 2023. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials.

Profil du candidat

The candidate should be in the last year of an engineering school or a master student (Bac+5) in a field related to machine learning/AI, who wishes to conduct research and development in an emerging, yet impactful field, in a collaborative environment. The intern will work in a team of researchers, post-docs, and PhD students who are actively investigating various challenges and aspects of federated learning. The candidate should have knowledge in machine learning and optimisation, and be skilled in Python programming and in using various machine learning libraries and frameworks. Prior knowledge in distributed systems are not mandatory, but appreciated.

Localisation du poste

Site

Saclay

Localisation du poste

France, Ile-de-France, Essonne (91)

Ville

Gif-sur-Yvette

Demandeur

Disponibilité du poste

01/03/2024