The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning (2111.05326v1)
Abstract: The Internet of Things (IoT) is on the verge of a major paradigm shift. In the IoT system of the future, IoFT, the cloud will be substituted by the crowd where model training is brought to the edge, allowing IoT devices to collaboratively extract knowledge and build smart analytics/models while keeping their personal data stored locally. This paradigm shift was set into motion by the tremendous increase in computational power on IoT devices and the recent advances in decentralized and privacy-preserving model training, coined as federated learning (FL). This article provides a vision for IoFT and a systematic overview of current efforts towards realizing this vision. Specifically, we first introduce the defining characteristics of IoFT and discuss FL data-driven approaches, opportunities, and challenges that allow decentralized inference within three dimensions: (i) a global model that maximizes utility across all IoT devices, (ii) a personalized model that borrows strengths across all devices yet retains its own model, (iii) a meta-learning model that quickly adapts to new devices or learning tasks. We end by describing the vision and challenges of IoFT in reshaping different industries through the lens of domain experts. Those industries include manufacturing, transportation, energy, healthcare, quality & reliability, business, and computing.
- Ford sync. https://www.ford.com/technology/sync/. Accessed: 2020-07-18.
- Vehicle-to-vehicle wireless power transfer: Paving the way toward an electrified transportation system. Transportation Research Part C: Emerging Technologies, 103:261–280, 2019.
- Federated learning based on dynamic regularization. In International Conference on Learning Representation, 2019.
- A real-time network-level traffic signal control methodology with partial connected vehicle information. Transportation Research Part C: Emerging Technologies, 121:102830, 2020.
- Federated learning: A survey on enabling technologies, protocols, and applications. IEEE Access, 8:140699–140725, 2020.
- Integrative density forecast and uncertainty quantification of wind power generation. to appear in IEEE Transactions on Sustainable Energy, 2021.
- Apple. Designing for privacy. https://developer.apple.com/videos/play/wwdc2019/708, 2019. Accessed: 2021-04-21.
- Federated evaluation of on-device personalization. arXiv preprint arXiv:1912.00818, 2019.
- The internet of things: A survey. Computer networks, 54(15):2787–2805, 2010.
- AWS. What is aws? https://www.youtube.com/watch?v=a9__D53WsUs, 2019. Accessed: 2020-07-18.
- AWS. Amazon web services (aws). https://aws.amazon.com/, 2021. Accessed: 2020-07-18.
- Azure. How does microsoft azure work? https://www.youtube.com/watch?v=KXkBZCe699A, 2018. Accessed: 2020-07-18.
- F. Bach. Consistency of the group Lasso and multiple kernel learning. Journal of Machine Learning Research, 9:1179–1225, 2008.
- How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2938–2948, 2020.
- Rényi fair inference. 2020.
- Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 2002.
- Discriminative k-shot learning using probabilistic models. arXiv preprint arXiv:1706.00326, 2017.
- Additive manufacturing, cloud-based 3d printing and associated services—overview. Journal of Manufacturing and Materials Processing, 1(2):15, 2017.
- How to achieve and sustain the impact of digital manufacturing at scale. McKinsey&Company Quartley, 2017.
- The power of synergy in differential privacy: Combining a small curator with local randomizers. arXiv preprint arXiv:1912.08951, 2019.
- Personalized and private peer-to-peer machine learning. In International Conference on Artificial Intelligence and Statistics, pages 473–481. PMLR, 2018.
- Evaluation of weather datasets for building energy simulation. Energy and Buildings, 49:109–118, 2012.
- Protection against reconstruction and its applications in private federated learning. arXiv preprint arXiv:1812.00984, 2018.
- P. J. Bickel and Y. Ritov. Non- and semiparametric statistics: compared and contrasted. J. Stat. Plan. Infer., 91:209–228, 2000.
- Integration of flexible consumers in the ancillary service markets. Energy, 67:479–489, 2014.
- Prochlo: Strong privacy for analytics in the crowd. In Proceedings of the 26th Symposium on Operating Systems Principles, pages 441–459, 2017.
- Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877, 2017.
- International electronic health record-derived covid-19 clinical course profiles: the 4ce consortium. Npj Digital Medicine, 3(1):1–9, 2020.
- Federated learning of predictive models from federated electronic health records. International journal of medical informatics, 112:59–67, 2018.
- Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning, 5(1):1–122, 2012.
- Cloud manufacturing: current trends and future implementations. Journal of Manufacturing Science and Engineering, 137(4), 2015.
- P. Buhlmann and S. van de Geer. Statistical for High-Dimensional Data. Springer Series in Statistics. Springer, New York, 2011.
- Adaptive learning in time-variant processes with application to wind power systems. IEEE Transactions on Automation Science and Engineering, 13(2):997–1007, 2015.
- A stochastic quasi-newton method for large-scale optimization. SIAM Journal on Optimization, 26(2):1008–1031, 2016.
- Decentralized control for residential energy management of a smart users’ microgrid with renewable energy exchange. IEEE/CAA Journal of Automatica Sinica, 6(3):641–656, 2019.
- Rich Caruana. Multitask learning. Machine learning, 28(1):41–75, 1997.
- Bayesian sequential design of experiments for extraction of single-crystal material properties from spherical indentation measurements on polycrystalline samples. JOM, 71(8):2671–2679, 2019.
- Distributed deep learning networks among institutions for medical imaging. Journal of the American Medical Informatics Association, 25(8):945–954, 2018.
- Federated meta-learning with fast convergence and efficient communication. arXiv preprint arXiv:1802.07876, 2018.
- Data mining for the internet of things: literature review and challenges. International Journal of Distributed Sensor Networks, 11(8):431047, 2015.
- Stochastic gradient descent in correlated settings: A study on gaussian processes. Advances in neural information processing systems, 2020a.
- Fedbe: Making bayesian model ensemble applicable to federated learning. International Conference on Learning Representation, 2021.
- Federated learning of out-of-vocabulary words. arXiv preprint arXiv:1903.10635, 2019a.
- Federated learning of n-gram language models. In ACL, 2019b.
- Random effects models for aggregate lifetime data. IEEE Transactions on Reliability, 66(1):76–83, 2016.
- Estimation of field reliability based on aggregate lifetime data. Technometrics, 59(1):115–125, 2017. 10.1080/00401706.2015.1096827. URL http://dx.doi.org/10.1080/00401706.2015.1096827.
- Parametric analysis of time-censored aggregate lifetime data. IISE Transactions, 52(5):516–527, 2020b.
- Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 15–26, 2017a.
- Sequential designs based on bayesian uncertainty quantification in sparse representation surrogate modeling. Technometrics, 59(2):139–152, 2017b.
- A closer look at few-shot classification. arXiv preprint arXiv:1904.04232, 2019c.
- Optimal client sampling for federated learning. arXiv preprint arXiv:2010.13723, 2020c.
- Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017c.
- Client selection in federated learning: Convergence analysis and power-of-choice selection strategies. arXiv preprint arXiv:2010.01243, 2020.
- Change-point detection on solar panel performance using thresholded lasso. Quality and Reliability Engineering International, 32(8):2653–2665, 2016.
- Managing data transfers in computer clusters with Orchestra. In ACM SIGCOMM, 2011.
- CNBC. Ford temporarily closes two plants after three workers test positive for coronavirus. https://www.cnbc.com/2020/05/20/ford-closes-chicago-plant-after-two-workers-test-positive-for-covid-19.html, 2020. Accessed: 2020-07-18.
- CNN. 12-year-old boy 3d prints masks for frontline workers. https://www.cnn.com/videos/us/2020/04/25/coronavirus-3d-print-ppe-12-year-old-pkg-whitfield-vpx.cnn, 2020. Accessed: 2020-07-18.
- P. L. Combettes. Monotone operator theory in convex optimization. Math. Program., 2018.
- Introduction to derivative-free optimization. SIAM, 2009.
- Rob Copeland. Google’s “project nightingale” gathers personal health data on millions of americans. https://www.wsj.com/articles/google-s-secret-project-nightingale-gathers-personal-health-data-on-millions-of-americans-11573496790, 2019. Accessed: 2021-04-25.
- The dial-a-ride problem: models and algorithms. Annals of operations research, 153(1):29–46, 2007.
- A new paradigm for organizing networks of computer numerical control manufacturing resources in cloud manufacturing. Procedia Manufacturing, 26:1318–1329, 2018.
- 3d printing enthusiasts are working from home to help hospitals fight coronavirus. https://www.cnn.com/2020/04/18/tech/us-coronavirus-ventilator-3d-printer-intl-hnk/index.html, 2020. Accessed: 2020-07-18.
- A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
- Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461, 2020.
- Electronic Parts Reliability Data 2014. Reliability Information Analysis Center, Rome, NY, 2014.
- Apple Differential Privacy Team. Learning with privacy at scale. In Apple Machine Learning Journal, 2017.
- Collecting telemetry data privately. In Advances in Neural Information Processing Systems, pages 3571–3580, 2017.
- ALEX: an updatable adaptive learned index. In ACM SIGMOD, 2020.
- Personalized federated learning with moreau envelopes. In 34th Conference on Neural Information Processing Systems, 2020.
- Jakub Dolezal. 3d printed face shields for medics and professionals - join us! https://forum.prusaprinters.org/forum/coronavirus-covid-19/3d-printed-face-shields-for-medics-and-professionals-join-us/, 2020. Accessed: 2020-07-18.
- PCC: Re-architecting congestion control for consistent high performance. In USENIX NSDI, 2015.
- P. Drineas and M. W. Mahoney. On the Nyström method for approximating a Gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6:2153–2175, 2005.
- Faster least squares approximation. Numerical Mathematics, 117:219–249, 2011.
- Fairness-aware agnostic federated learning. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 181–189. SIAM, 2021.
- A limited-preview filtered b-spline approach to tracking control–with application to vibration-induced error compensation of a 3d printer. Mechatronics, 56:287–296, 2018.
- Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications. In 2019 IEEE 37th International Conference on Computer Design (ICCD), pages 246–254. IEEE, 2019.
- Towards a neural statistician. International Conference on Learning Representations, 2017.
- EIA. U.S energy information administration, U.S energy facts explained. https://www.eia.gov/energyexplained/us-energy-facts/, 2020a. Accessed: 2021-04-16.
- EIA. U.S energy information administration, today in energy. https://www.eia.gov/todayinenergy/detail.php?id=42635, 2020b. Accessed: 2021-03-29.
- EIA. U.S energy information administration, consumption and efficiency. https://www.eia.gov/consumption/, 2021. Accessed: 2021-04-16.
- Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33, 2020.
- Covid-resnet: A deep learning framework for screening of covid19 from radiographs. arXiv preprint arXiv:2003.14395, 2020.
- FDA. 3d printing in fda’s rapid response to covid-19. https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/3d-printing-fdas-rapid-response-covid-19, 2020. Accessed: 2020-07-18.
- Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 259–268, 2015.
- Estimation of lead vehicle kinematics using camera-based data for driver distraction detection. International Journal of Automotive Engineering, 9(3):158–164, 2018.
- Karen Field. Covid-19: How quickly can manufacturing respond to the surge in demand? https://www.fierceelectronics.com/electronics/how-quickly-can-manufacturing-respond-to-surge-demand, 2000. Accessed: 2020-07-18.
- Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126–1135. PMLR, 2017.
- Probabilistic model-agnostic meta-learning. Advances In Neural Information Processing Systems, 2018.
- General motors relies on iot to anticipate customers needs. https://sloanreview.mit.edu/article/general-motors-relies-on-iot-to-keep-its-customers-safe-and-secure/, 2016. Accessed: 2020-07-18.
- Assessment of demand response and advanced metering. Technical report, Federal Energy Regulatory Commission, 2017.
- The elements of statistical learning, volume 1. Springer series in statistics New York, 2001.
- Sustainable manufacturing with cyber-physical discrete manufacturing networks: Overview and modeling framework. Journal of Manufacturing Science and Engineering, 141(2), 2019.
- Conditional neural processes. In International Conference on Machine Learning, pages 1704–1713. PMLR, 2018a.
- Neural processes. arXiv preprint arXiv:1807.01622, 2018b.
- Online learning multipath transport. In ACM CoNEXT, 2020.
- Multi-task sparse structure learning with gaussian copula models. Journal of Machine Learning Research, 2016.
- Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
- Google. Your chats stay private while messages improves suggestions. https://support.google.com/messages/answer/9327902?hl=en#zippy=, 2019. Accessed: 2021-04-21.
- GoogleCloud. Googlecloud. https://cloud.google.com/solutions/iot, 2021. Accessed: 2020-07-18.
- Meta-learning probabilistic inference for prediction. arXiv preprint arXiv:1805.09921, 2018.
- Y. Gordon. Some inequalities for Gaussian processes and applications. Israel Journal Math., 50:265–289, 1985.
- Robert B Gramacy. Surrogates: Gaussian process modeling, design, and optimization for the applied sciences. CRC Press, 2020.
- Recasting gradient-based meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930, 2018.
- Federated learning of a mixture of global and local models. arXiv preprint arXiv:2002.05516, 2021.
- Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604, 2018.
- Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315–3323, 2016.
- Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677, 2017.
- Federated learning for ranking browser history suggestions. In arxiv.org/abs/1911.11807, 2019.
- Statistical Learning with Sparsity: The Lasso and Generalizations. Monographs on Statistics and Applied Probability 143. CRC Press, New York, 2015.
- Feasibility of connecting machinery and robots to industrial control services in the cloud. In 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA), pages 1–4. IEEE, 2016.
- Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439, 2020.
- Internet of things (iot) and the energy sector. Energies, 13(2):494, 2020.
- Robust optimal energy management of a residential microgrid under uncertainties on demand and renewable power generation. IEEE Transactions on Automation Science and Engineering, 18(2):618–637, 2021.
- Focus: Querying large video datasets with low latency and low cost. In USENIX OSDI, 2018.
- Fedmgda+: Federated learning meets multi-objective optimization. arXiv preprint arXiv:2006.11489, 2020.
- Loadaboost: Loss-based adaboost federated machine learning with reduced computational complexity on iid and non-iid intensive care data. Plos one, 15(4):e0230706, 2020a.
- Loadaboost: loss-based adaboost federated machine learning with reduced computational complexity on iid and non-iid intensive care data. arXiv preprint arXiv:1811.12629, 2020b.
- Fairness and accuracy in federated learning. arXiv preprint arXiv:2012.10069, 2020c.
- Analysis of computer experiments with functional response. Technometrics, 57(1):35–44, 2015.
- Training neural networks using features replay. In Advances in Neural Information Processing Systems, pages 6659–6668, 2018.
- Mfbo-ssm: Multi-fidelity bayesian optimization for fast inference in state-space models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7858–7865, 2019.
- Rishi Iyengar. Can 3d printing plug the coronavirus equipment gap? https://www.cnn.com/2020/04/16/tech/coronavirus-medical-equipment-3d-printing/index.html, 2020. Accessed: 2020-07-18.
- Averaging weights leads to wider optima and better generalization. uncertainty in artificial intelligence, 2018.
- Projecting the most likely annual urban heat extremes in the central united states. Atmosphere, 10(12):727, 2019.
- Probabilistic characterization of wind diurnal variability for wind resource assessment. IEEE Transactions on Sustainable Energy, 11(4):2535–2544, 2020.
- On the long-term density prediction of peak electricity load with demand side management in buildings. Energy and Buildings, 228:110450, 2020.
- A deep reinforcement learning perspective on Internet congestion control. In International Conference on Machine Learning, 2019.
- Chameleon: scalable adaptation of video analytics. In ACM SIGCOMM, pages 253–266, 2018.
- Networked cameras are the new big data clusters. In HotEdgeVideo, 2019a.
- Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv:1909.12488, 2019b.
- Space-filling designs for robustness experiments. Technometrics, 61(1):24–37, 2019.
- Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977, 2019.
- Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory. IEEE Internet of Things Journal, 6(6):10700–10714, 2019.
- Design and modeling strategies for mixture-of-mixtures experiments. Technometrics, 53(2):125–136, 2011a.
- Learning with whom to share in multi-task feature learning. In ICML, 2011b.
- Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pages 5132–5143. PMLR, 2020.
- Not all samples are created equal: Deep learning with importance sampling. Proceedings of the 35 th International Conference on Machine Learning, (80), 2018.
- Ai-facilitated health care requires education of clinicians. The Lancet, 397(10281):1254, 2021.
- Want to scale in centralized systems? think p2p. Journal of Internet Services and Applications, 6(1):1–12, 2015.
- Attentive neural processes. International Conference on Learning Representations, 2019.
- Powell Kimberly. Nvidia clara federated learning to deliver ai to hospitals while protecting patient data. https://blogs.nvidia.com/blog/2019/12/01/clara-federated-learning/, 2019. Accessed: 2021-04-21.
- Adam: A method for stochastic optimization. International Conference on Learning Representations, 2015.
- Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
- Full-load route planning for balancing bike sharing systems by logic-based benders decomposition. Networks, 69(3):270–289, 2017.
- Probabilistic graphical models: principles and techniques. MIT press, 2009.
- Decentralized deep learning with arbitrary communication compression. arXiv preprint arXiv:1907.09356, 2019.
- V. Koltchinskii and M. Yuan. Sparsity in multiple kernel learning. Annals of Statistics, 38:3660–3695, 2010.
- Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527, 2016.
- Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.
- Minimizing negative transfer of knowledge in multivariate gaussian processes: A scalable and regularized approach. IEEE Transactions of Pattern Analysis and Machine Intelligence. To appear.
- Nonparametric-condition-based remaining useful life prediction incorporating external factors. IEEE Transactions on Reliability, 67(1):41–52, 2017.
- Nonparametric modeling and prognosis of condition monitoring signals using multivariate gaussian convolution processes. Technometrics, 60(4):484–496, 2018.
- Minimizing negative transfer of knowledge in multivariate gaussian processes: A scalable and regularized approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
- The case for learned index structures. In ACM SIGMOD, 2018.
- Survey of personalization techniques for federated learning. In 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), pages 794–797, 2020. 10.1109/WorldS450073.2020.9210355.
- Learning task grouping and overlap in multi-task learning. arXiv preprint arXiv:1206.6417, 2012.
- Implementing the smart factory: New perspectives for driving value. Deloitte Insights, Deloitte, USA, 2020.
- Deep prior. arXiv preprint arXiv:1712.05016, 2017.
- Uncertainty in multitask transfer learning. arXiv preprint arXiv:1806.07528, 2018.
- The theory of incentives: the principal-agent model. Princeton University press, Princeton, NJ, 2009.
- Sol: A federated execution engine for fast distributed computation over slow networks. In USENIX NSDI, 2020a.
- Oort: Informed participant selection for scalable federated learning. arXiv preprint arXiv:2010.06081, 2020b.
- A distributed framework for coordinated heavy-duty vehicle platooning. IEEE Transactions on Intelligent Transportation Systems, 16(1):419–429, 2014.
- Matt Leonard. With predictive maintenance, operators seek improved uptime. https://www.supplychaindive.com/news/with-predictive-maintenance-operators-seek-improved-uptime/561684/, 2019a. Accessed: 2020-07-18.
- Matt Leonard. Declining price of iot sensors means greater use in manufacturing. https://www.supplychaindive.com/news/declining-price-iot-sensors-manufacturing/564980/, 2019b. Accessed: 2020-07-18.
- H Leurent and E De Boer. The next economic growth engine: Scaling fourth industrial revolution technologies in production. In World Economic Forum, 2018.
- Heat - human embodied autonomous thermostat. Building and Environment, 178:106879, 2020a.
- Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. In The World Wide Web Conference, pages 983–994, 2019a.
- On negative transfer and structure of latent functions in multi-output gaussian processes. arXiv preprint arXiv:2004.02382, 2020.
- Federated optimization in heterogeneous networks. Proceedings of the 3rd MLSys Conference, 2018.
- Feddane: A federated newton-type method. In 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pages 1227–1231. IEEE, 2019b.
- Fair resource allocation in federated learning. arXiv preprint arXiv:1905.10497, 2019c.
- Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60, 2020b.
- Ditto: Fair and robust federated learning through personalization. arXiv preprint arXiv:2012.04221, 2021.
- Privacy-preserving federated brain tumour segmentation. In Machine Learning in Medical Imaging, 2019d.
- Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835, 2017.
- Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. arXiv preprint arXiv:1705.09056, 2017.
- Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523, 2020.
- Federated learning in mobile edge networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 22(3):2031–2063, 2020.
- Real-time edge intelligence in the making: A collaborative learning framework via federated meta-learning. arXiv preprint arXiv:2001.03229, 2020.
- The nonparanormal: Semiparametric estimation of high-dimensional undirected graphs. Journal of Machine Learning Research, 10:1–37, 2009.
- Diagnosing multistage manufacturing processes with engineering-driven factor analysis considering sampling uncertainty. Journal of Manufacturing Science and Engineering, 135(4), 2013.
- Adaptive sensor allocation strategy for process monitoring and diagnosis in a bayesian network. IEEE Transactions on Automation Science and Engineering, 11(2):452–462, 2013.
- Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances In Neural Information Processing Systems, 2016.
- Adam with bandit sampling for deep learning. NeurIPS, 2020a.
- Accelerating federated learning via momentum gradient descent. IEEE Transactions on Parallel and Distributed Systems, 31(8):1754–1766, 2020b.
- Trojaning attack on neural networks. 2017.
- Optimal operation of independent regional power grid with multiple wind-solar-hydro-battery power. Applied Energy, 235:1541–1550, 2019. ISSN 0306-2619. https://doi.org/10.1016/j.apenergy.2018.11.072. URL https://www.sciencedirect.com/science/article/pii/S0306261918317781.
- The functional neural process. Advances in Neural Information Processing Systems, 2019.
- Deep coupled resnet for low-resolution face recognition. IEEE Signal Processing Letters, 25(4):526–530, 2018. 10.1109/LSP.2018.2810121.
- Real-world image datasets for federated learning. In arxiv.org/abs/1910.11089, 2019.
- Qi Luo and Romesh Saigal. Dynamic multiagent incentive contracts - existence, uniqueness and implementation. Mathematics, 9(1), 2021. 10.3390/math9010019.
- Threats to federated learning: A survey. arXiv preprint arXiv:2003.02133, 2020.
- Variational implicit processes. In International Conference on Machine Learning, pages 4222–4233. PMLR, 2019.
- Internet of things (iot): A literature review. Journal of Computer and Communications, 3(05):164, 2015.
- A simple baseline for bayesian uncertainty in deep learning. Advances in Neural Information Processing Systems, 32:13153–13164, 2019.
- Remotely controlled manufacturing: A new frontier for systems research. In Proceedings of the 21st International Workshop on Mobile Computing Systems and Applications, pages 62–67, 2020.
- Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
- Nonelectronic Parts Reliability Data 2011. Reliability Information Analysis Center, Rome, NY, 2011.
- M. W. Mahoney and P. Drineas. CUR matrix decompositions for improved data analysis. Proc. Natl. Acad. Sci. USA, 106(3):697–702, 2009.
- Renewable electricity futures study. executive summary. Technical report, National Renewable Energy Lab.(NREL), Golden, CO (United States), 2012.
- Surface heat assessment for developed environments: Probabilistic urban temperature modeling. Computers, Environment and Urban Systems, 66:53–64, 2017.
- Three approaches for personalization with applications to federated learning. CoRR, abs/2002.10619, 2020. URL https://arxiv.org/abs/2002.10619.
- Neural adaptive video streaming with Pensieve. In ACM SIGCOMM, 2017.
- Neda Masoud and R Jayakrishnan. A decomposition algorithm to solve the multi-hop peer-to-peer ride-matching problem. Transportation Research Part B: Methodological, 99:1–29, 2017.
- Efficient large-scale distributed training of conditional maximum entropy models. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 22. Curran Associates, Inc., 2009. URL https://proceedings.neurips.cc/paper/2009/file/d81f9c1be2e08964bf9f24b15f0e4900-Paper.pdf.
- McKinsey. The age of analytics: competing in a data-driven world, 2016.
- Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pages 1273–1282, 2017a.
- Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. JMLR, 2017b.
- H Brendan McMahan et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1), 2021.
- Demand response for variable renewable energy integration: A proposed approach and its impacts. Energy, 197:117205, 2020.
- Statistical Methods for Reliability Data. John Wiley & Sons, 1998.
- Microsoft. 2019 manufacturing trends report. https://info.microsoft.com/rs/157-GQE-382/images/EN-US-CNTNT-Report-2019-Manufacturing-Trends.pdf, 2019. Accessed: 2020-07-18.
- Optimal network-level traffic signal control: A benders decomposition-based solution algorithm. Transportation Research Part B: Methodological, 121:252–274, 2019.
- Agnostic federated learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4615–4625. PMLR, 09–15 Jun 2019a.
- Agnostic federated learning. In International Conference on Machine Learning, pages 4615–4625. PMLR, 2019b.
- Nicole Casal Moore. 3-d printing gets a turbo boost from u-m technology. https://news.umich.edu/3-d-printing-gets-a-turbo-boost-from-u-m-technology/, November 2017. Accessed: 2019-02-19.
- Delay mitigation in offloaded cloud controllers in industrial iot. IEEE Access, 5:4418–4430, 2017.
- Meta networks. In International Conference on Machine Learning, pages 2554–2563. PMLR, 2017.
- The role and impact of industry 4.0 and the internet of things on the business strategy of the value chain—the case of hungary. Sustainability, 10(10):3491, 2018.
- Radaptive sampling probabilities for non-smooth optimizatio. Proceedings of the 34 th International Conference on Machine Learning, (70), 2017.
- Optimal residential community demand response scheduling in smart grid. Applied Energy, 210:1280–1289, 2018.
- Grid-interactive efficient buildings. Technical report, US Dept. of Energy (USDOE), Washington DC (United States); Navigant …, 2019.
- Pac-bayesian meta-learning with implicit prior. IEEE TRANSACTION ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020a.
- Deep learning methods in transportation domain: a review. IET Intelligent Transport Systems, 12(9):998–1004, 2018.
- Resource allocation in mobility-aware federated learning networks: a deep reinforcement learning approach. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT), pages 1–6. IEEE, 2020b.
- Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2(2):1, 2018.
- On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018.
- Federated learning for wireless communications: Motivation, opportunities, and challenges. IEEE Communications Magazine, 58(6):46–51, 2020.
- A sparse partitioned-regression model for nonlinear system–environment interactions. IISE Transactions, 49(8):814–826, 2017.
- NOAA. Climate change: Global temperature. https://www.climate.gov/news-features/understanding-climate/climate-change-global-temperature, 2021. Accessed: 2021-04-17.
- Entity resolution and federated learning get a federated resolution. arXiv preprint arXiv:1803.04035, 2018.
- OCR. Office for civil rights, research. https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/research/index.html, 2009. Accessed: 2021-04-25.
- A three-tier redundant architecture for safe and reliable cloud-based cnc over public internet networks. Robotics and Computer-Integrated Manufacturing, 62:101880, 2020.
- Distributed manufacturing for and by the masses. In Science, 2021.
- Low-level control of 3d printers from the cloud: A step toward 3d printer control as a service. Inventions, 3(3):56, 2018.
- OnStar. Welcome to onstar. https://www.onstar.com/, 2021. Accessed: 2020-07-18.
- OREDA. OREDA Offshore Reliability Data Handbook. Det Norske Veritas (DNV), 2009.
- On the assessment and control optimisation of demand response programs in residential buildings. Renewable and Sustainable Energy Reviews, 127:109861, 2020.
- A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
- A survey on transfer learning ieee transactions on knowledge and data engineering. 22 (10): 1345, 1359, 2010.
- Learning large-scale poisson dag models based on overdispersion scoring. In Advances in Neural Information Processing Systems, pages 631–639, 2015.
- Deep kernel transfer in gaussian processes for few-shot learning. arXiv preprint arXiv:1910.05199, 2019.
- Bayesian meta-learning for the few-shot setting via deep kernels. Advances in Neural Information Processing Systems, 2020.
- Fedsplit: an algorithmic framework for fast federated optimization. In 34th Conference on Neural Information Processing Systems, 2020. URL https://proceedings.neurips.cc/paper/2020/file/4ebd440d99504722d80de606ea8507da-Paper.pdf.
- D. W. Peaceman and Jr. H. H. Rachford. The numerical solution of parabolic and elliptic differential equations. Journal of the SIAM, 1955.
- Judea Pearl. Causality: models, reasoning and inference, volume 29. Cambridge Univ Press, 2000.
- A partition-based match making algorithm for dynamic ridesharing. IEEE Transactions on Intelligent Transportation Systems, 16(5):2587–2598, 2015.
- Bayesian deep-learning-based health prognostics toward prognostics uncertainty. IEEE Transactions on Industrial Electronics, 67(3):2283–2293, 2019.
- Public trust in health information sharing: a measure of system trust. Health services research, 53(2):824–845, 2018.
- Matthew Plumlee. Bayesian calibration of inexact computer models. Journal of the American Statistical Association, 112(519):1274–1285, 2017.
- Matthew Plumlee. Computer model calibration with confidence and consistency. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 81(3):519–545, 2019.
- Convex multi-task feature learning. Journal of Machine Learning, 10:243–272, 2007.
- How smart, connected products are transforming competition. Harvard business review, 92(11):64–88, 2014.
- Low latency geo-distributed data analytics. In ACM SIGCOMM, 2015.
- Peihua Qiu. Big data? statistical process control can help! The American Statistician, 74(4):329–344, 2020.
- Language models are unsupervised multitask learners. OpenAI, 2019.
- A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE Internet of Things Journal, 2020.
- Federated learning for emoji prediction in a mobile keyboard. arXiv preprint arXiv:1906.04329, 2019.
- G. Raskutti and M. Mahoney. A statistical perspective on randomized sketching for ordinary least-squares. Journal of Machine Learning Research, 17(213):1–31, 2016.
- Minimax-optimal rates for sparse additive models over kernel classes via convex programming. Journal of Machine Learning Research, 13:398–427, 2012.
- Early stopping and non-parametric regression: An optimal data-dependent stopping rule. Journal of Machine Learning Research, 15:335–366, 2014.
- C. E. Rasmussen and C. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006.
- Amortized bayesian meta-learning. In ICLR (Poster), 2019.
- Optimization as a model for few-shot learning. International Conference on Learning Representations, 2017.
- High-dimensional covariance estimation by minimizing ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-penalized log-determinant divergence. Electronic Journal of Statistics, 5:935–980, 2011.
- Adaptive methods for nonconvex optimization. In Proceeding of 32nd Conference on Neural Information Processing Systems (NIPS 2018), 2018.
- Adaptive federated optimization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=LkFG3lB13U5.
- Robust and communication-efficient collaborative learning. In Advances in Neural Information Processing Systems, pages 8388–8399, 2019.
- Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3):1247–1293, 2013.
- Rockwell. The connected enterprise. https://www.rockwellautomation.com/en-us/capabilities/connected-enterprise.html, 2021. Accessed: 2020-07-18.
- Meta-learning with latent embedding optimization. International Conference on Learning Representations, 2019.
- Variable generation, reserves, flexibility and policy interactions. In 2014 47th Hawaii International Conference on System Sciences, pages 2426–2434. IEEE, 2014.
- Primer on monotone operator methods. In Appl. Comput. Math., 16. URL https://stanford.edu/~boyd/papers/pdf/monotone_primer.pdf.
- Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017.
- 3d printing in covid-19: Productivity estimation of the most promising open source solutions in emergency situations. Applied Sciences, 10(11):4004, 2020.
- Distributed federated learning for ultra-reliable low-latency vehicular communications. IEEE Transactions on Communications, 68(2):1146–1159, 2019.
- Samsung. Samsung galaxy watch active 2. https://www.samsung.com/us/mobile/wearables/galaxy-watch-active-2/, 2021. Accessed: 2020-07-18.
- Yuliy Sannikov. A continuous-time version of the principal-agent problem. Rev. Econ. Stud., 75(3):957–984, 2008.
- Federated learning improves site performance in multicenter deep learning without data sharing. Journal of the American Medical Informatics Association, 2021.
- Clustered federated learning: Model-agnostic distributed multi-task optimization under privacy constraints. arXiv preprint arXiv:1910.01991, 2019.
- Distributed demand side management with stochastic wind power forecasting. IEEE Transactions on Control Systems Technology, pages 1–16, 2021.
- Minimizing finite sums with the stochastic average gradient. arXiv preprint arXiv:1309.2388, 2015.
- First order generative adversarial networks. International Conference on Machine Learning, 2018.
- Communication-efficient distributed optimization using an approximate newton-type method. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 1000–1008. PMLR, 2014.
- Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Scientific reports, 10(1):1–12, 2020.
- Fed-ensemble: Ensemble models in federated learning for improved generalization and uncertainty quantification. In Arxiv:2107.10663, 2021.
- Detailed comparison of communication efficiency of split learning and federated learning. arXiv preprint arXiv:1909.09145, 2019.
- Evaluating a widely implemented proprietary deterioration index model among hospitalized covid-19 patients. Annals of the American Thoracic Society, (ja), 2020.
- Federated multi-task learning. In 31st Conference on Neural Information Processing Systems, 2017a.
- Federated multi-task learning. NeurIPS, 2017b.
- Federated multi-task learning. Conference on Neural Information Processing Systems, 2017c.
- Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175, 2017.
- On repeated moral hazard with discounting. Rev. Econ. Stud., 54(4):599–617, 1987.
- Distributed manufacturing: scope, challenges and opportunities. International Journal of Production Research, 54(23):6917–6935, 2016.
- Robest Stevens. Why diy 3d-printed face masks and shields are so risky. https://slate.com/technology/2020/04/diy-3d-printed-face-masks-shields-coronavirus.html, 2020. Accessed: 2020-07-18.
- Alice Su. Faulty masks. flawed tests. china’s quality control problem in leading global covid-19 fight. https://www.latimes.com/world-nation/story/2020-04-10/china-beijing-supply-world-coronavirus-fight-quality-control, 2020. Accessed: 2020-07-18.
- Machine Learning in Non-Stationary Environments. MIT Press, 2012.
- How to fine-tune bert for text classification? Chinese Computational Linguistics, 2019.
- Trip-based graph partitioning in dynamic ridesharing. Transportation Research Part C: Emerging Technologies, 114:532–553, 2020.
- Doublesqueeze: Parallel stochastic gradient descent with double-pass error-compensated compression. In International Conference on Machine Learning, pages 6155–6165, 2019.
- Adaptive deep kernel learning. arXiv preprint arXiv:1905.12131, 2019.
- Learning to route. In ACM HotNets, 2017.
- Joaquin Vanschoren. Meta-learning: A survey. arXiv preprint arXiv:1810.03548, 2018.
- Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
- De Brouwer Walter. The federated future is ready for shipping. https://medium.com/@_doc_ai/the-federated-future-is-ready-for-shipping-d17ff40f43e3, 2019. Accessed: 2021-04-21.
- Federated learning with matched averaging. International Conference on Learning Representation, 2020.
- Look-ahead planning for renewable energy: A dynamic “predict and store” approach. submitted, 2021a.
- Distribution inference from early-stage stationary data streams by transfer learning. IISE Transactions, pages 1–25, 2021b.
- Federated learning with personalization layers. arXiv preprint arXiv:1910.10252, 2019a.
- Federated evaluation of on-device personalization. arXiv preprint arXiv:1910.10252, 2019b.
- In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Network, 33(5):156–165, 2019c.
- Bayesian meta sampling for fast uncertainty adaptation. In International Conference on Learning Representations, 2019d.
- Griffin M Weber. Federated queries of clinical data repositories: scaling to a national network. Journal of biomedical informatics, 55:231–236, 2015.
- The shared health research information network (shrine): a prototype federated query tool for clinical data repositories. Journal of the American Medical Informatics Association, 16(5):624–630, 2009.
- Colight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1913–1922, 2019a.
- Federated learning with differential privacy: Algorithms and performance analysis. arXiv preprint arXiv:1911.00222, 2019b.
- Reliability and reliability-based importance analysis of structural systems using multiple response gaussian process model. Reliability Engineering & System Safety, 175:183–195, 2018.
- Paul Wellener et al. Deloitte and mapi smart factory study: capturing value through the digital journey. Deloitte Insights and MAPI, Deloitte, USA, 2019.
- Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 681–688, 2011.
- External validations of cardiovascular clinical prediction models: A large-scale review of the literature. medRxiv, 2021.
- Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
- TCP ex machina: Computer-generated congestion control. In ACM SIGCOMM, 2013.
- Probast: a tool to assess the risk of bias and applicability of prediction model studies. Annals of internal medicine, 170(1):51–58, 2019.
- Christina Wong. Ai chips for self driving cars will a be 10 billion market by 2024. https://www.nextbigfuture.com/2019/03/ai-chips-for-self-driving-cars-will-a-be-10-billion-market-by-2024.html, 2019. Accessed: 2020-07-18.
- An overview of george box’s contributions to process monitoring and feedback adjustment. Applied Stochastic Models in Business and Industry, 30(1):53–61, 2014.
- Experiments: planning, analysis, and optimization, volume 552. John Wiley & Sons, 2011.
- Fast-convergent federated learning with adaptive weighting. arXiv preprint arXiv:2012.00661, 2020.
- Meta-learning autoencoders for few-shot prediction. arXiv preprint arXiv:1807.09912, 2018.
- Optimizing latin hypercube design for sequential sampling of computer experiments. Engineering Optimization, 41(8):793–810, 2009.
- Client selection and bandwidth allocation in wireless federated learning networks: A long-term perspective. IEEE Transactions on Wireless Communications, 2020.
- Condition monitoring of wind power system with nonparametric regression analysis. IEEE Transactions on Energy Conversion, 29(2):288–299, 2014.
- Learning in situ: a randomized experiment in video streaming. In USENIX NSDI, 2020.
- Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019a.
- Parallel distributed logistic regression for vertical federated learning without third-party coordinator. arXiv preprint arXiv:1911.09824, 2019b.
- Applied federated learning: Improving Google keyboard query suggestions. In arxiv.org/abs/1812.02903, 2018a.
- Applied federated learning: Improving google keyboard query suggestions. arXiv preprint arXiv:1812.02903, 2018b.
- The unusual effectiveness of averaging in gan training. International Conference on Learning Representations, 2019.
- How do heterogeneities in operating environments affect field failure predictions and test planning? The Annals of Applied Statistics, 7:2249–2271, 2013.
- Bayesian model-agnostic meta-learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 7343–7353, 2018.
- When wind travels through turbines: A new statistical approach for characterizing heterogeneous wake effects in multi-turbine wind farms. IISE Transactions, 49(1):84–95, 2017.
- Salvaging federated learning by local adaptation. arXiv preprint arXiv:2002.04758, 2020.
- Federated accelerated stochastic gradient descent. In 34th Conference on Neural Information Processing Systems, 2020. URL https://papers.nips.cc/paper/2020/file/39d0a8908fbe6c18039ea8227f827023-Supplemental.pdf.
- M. Yuan and Y. Lin. Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1):19–35, 2007.
- Visual classification with multitask joint sparse representation. IEEE Transactions on Image Processing, 21(10):4349–4360, 2012.
- The renyi gaussian process: Towards improved generalization. arXiv preprint arXiv:1910.06990, 2019a.
- Variational inference of joint models using multivariate gaussian convolution processes. arXiv preprint arXiv:1903.03867, 2019b.
- Gifair-fl: An approach for group and individual fairness in federated learning. arXiv preprint arXiv:2108.02741, 2021.
- Bayesian nonparametric federated learning of neural networks. In International Conference on Machine Learning, pages 7252–7261. PMLR, 2019.
- Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pages 962–970. PMLR, 2017.
- Improving fairness via federated learning. arXiv preprint arXiv:2110.15545, 2021.
- AWStream: Adaptive wide-area streaming analytics. In ACM SIGCOMM, 2018a.
- Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence, 41(8):2008–2026, 2018b.
- Fairfl: A fair federated learning approach to reducing demographic bias in privacy-sensitive classification models. In 2020 IEEE International Conference on Big Data (Big Data), pages 1051–1060. IEEE, 2020a.
- Enabling industrial internet of things (iiot) towards an emerging smart energy system. Global energy interconnection, 1(1):39–47, 2018c.
- Mi Zhang. Federated learning: The future of distributed machine learning. https://medium.com/syncedreview/federated-learning-the-future-of-distributed-machine-learning-eec95242d897, 2019. Accessed: 2020-07-18.
- On the connection between feddyn and fedpd, 2021. URL http://people.ece.umn.edu/~mhong/FedDyn_FedPD.pdf.
- Fedpd: A federated learning framework with optimal rates and adaptivity to non-iid data, 2020b.
- Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning. arXiv preprint arXiv:1203.3536, 2012.
- Ridesharing problem with flexible pickup and delivery locations for app-based transportation service: mathematical modeling and decomposition methods. Journal of Advanced Transportation, 2018, 2018a.
- Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018b.
- Energy internet: the business perspective. Applied Energy, 178:212–222, 2016.
- Deep leakage from gradients. 33rd Conference on Neural Information Processing Systems, 2019.
- Acute physiology and chronic health evaluations (apache) iv: hospital mortality assessment for today’s critically ill patients. Crit Care Med, 34(5):1297–1310, 2006.
- Parallelized stochastic gradient descent. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010. URL https://proceedings.neurips.cc/paper/2010/file/abea47ba24142ed16b7d8fbf2c740e0d-Paper.pdf.
- Evaluating the contribution of energy storages to support large-scale renewable generation in joint energy and ancillary service markets. IEEE Transactions on Sustainable Energy, 7(2):808–818, 2016.
- Gradient-em bayesian meta-learning. Advances in Neural Information Processing Systems, 2020.
- Raed Kontar (6 papers)
- Naichen Shi (11 papers)
- Xubo Yue (13 papers)
- Seokhyun Chung (8 papers)
- Eunshin Byon (6 papers)
- Mosharaf Chowdhury (39 papers)
- Judy Jin (4 papers)
- Wissam Kontar (9 papers)
- Neda Masoud (11 papers)
- Maher Noueihed (1 paper)
- Chinedum E. Okwudire (6 papers)
- Garvesh Raskutti (35 papers)
- Romesh Saigal (6 papers)
- Karandeep Singh (10 papers)
- Zhisheng Ye (8 papers)