Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design (2306.14348v2)

Published 25 Jun 2023 in cs.LG and cs.DC

Abstract: Optimal design is a critical yet challenging task within many applications. This challenge arises from the need for extensive trial and error, often done through simulations or running field experiments. Fortunately, sequential optimal design, also referred to as Bayesian optimization when using surrogates with a Bayesian flavor, has played a key role in accelerating the design process through efficient sequential sampling strategies. However, a key opportunity exists nowadays. The increased connectivity of edge devices sets forth a new collaborative paradigm for Bayesian optimization. A paradigm whereby different clients collaboratively borrow strength from each other by effectively distributing their experimentation efforts to improve and fast-track their optimal design process. To this end, we bring the notion of consensus to Bayesian optimization, where clients agree (i.e., reach a consensus) on their next-to-sample designs. Our approach provides a generic and flexible framework that can incorporate different collaboration mechanisms. In lieu of this, we propose transitional collaborative mechanisms where clients initially rely more on each other to maneuver through the early stages with scant data, then, at the late stages, focus on their own objectives to get client-specific solutions. Theoretically, we show the sub-linear growth in regret for our proposed framework. Empirically, through simulated datasets and a real-world collaborative sensor design experiment, we show that our framework can effectively accelerate and improve the optimal design process and benefit all participants.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. Optimal experimental designs in the flow rate of particles. Technometrics, 58(2):269–276.
  2. Antony, J. (2014). Design of experiments for engineers and scientists. Elsevier.
  3. Batch bayesian optimization via simulation matching. Advances in Neural Information Processing Systems, 23.
  4. Hybrid batch bayesian optimization. arXiv preprint arXiv:1202.5597.
  5. Biosensors acting in injection mode. Mathematical Modeling of Biosensors, pages 183–205.
  6. Bellman, R. (1956). A problem in the sequential design of experiments. Sankhyā: The Indian Journal of Statistics (1933-1960), 16(3/4):221–229.
  7. Balancing communication and computation in distributed optimization. IEEE Transactions on Automatic Control, 64(8):3141–3155.
  8. On the experimental designs for exploring response surfaces. Ann Math Stat, 13:1–45.
  9. Metal 3d printing in construction: A review of methods, research, applications, opportunities and challenges. Engineering Structures, 180:332–348.
  10. Blueprints for biosensors: design, limitations, and applications. Genes, 9(8):375.
  11. Electrochemical biosensors for pathogen detection. Biosensors and Bioelectronics, 159:112214.
  12. Chernoff, H. (1959). Sequential design of experiments. The Annals of Mathematical Statistics, 30(3):755–770.
  13. Gaussian process optimization with mutual information. In International Conference on Machine Learning, pages 253–261. PMLR.
  14. Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association, 86(416):953–963.
  15. Federated bayesian optimization via thompson sampling. Advances in Neural Information Processing Systems, 33:9687–9699.
  16. Differentially private federated bayesian optimization with distributed exploration. Advances in Neural Information Processing Systems, 34:9125–9139.
  17. Sliced full factorial-based latin hypercube designs as a framework for a batch sequential design algorithm. Technometrics, 59(1):11–22.
  18. Bayesian optimization in ab initio nuclear physics. Journal of Physics G: Nuclear and Particle Physics, 46(9):095101.
  19. Sequential design for functional calibration of computer models. Technometrics, 60(3):286–296.
  20. Planning experiments seeking maxima. Techniques of statistical analysis, pages 365–372.
  21. Adaptive design and analysis of supercomputer experiments. Technometrics, 51(2):130–145.
  22. Optimization of multi-fidelity computer experiments via the eqie criterion. Technometrics, 59(1):58–68.
  23. Bayesian optimization of functional output in inverse problems. Optimization and Engineering, 22:2553–2574.
  24. Hunt, N. (2020). Batch Bayesian optimization. PhD thesis, Massachusetts Institute of Technology.
  25. Binoculars for efficient, nonmyopic sequential experimental design. In International Conference on Machine Learning, pages 4794–4803. PMLR.
  26. Sample preparation-free, real-time detection of microrna in human serum using piezoelectric cantilever biosensors at attomole level. Analytical chemistry, 84(23):10426–10436.
  27. Biosensor-based microrna detection: techniques, design, performance, and challenges. Analyst, 139(7):1576–1588.
  28. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455.
  29. Kleijnen, J. P. (2018). Design and analysis of simulation experiments. Springer.
  30. Bayesian treed multivariate gaussian process with adaptive design: Application to a carbon capture unit. Technometrics, 56(2):145–158.
  31. The internet of federated things (ioft). IEEE Access, 9:156071–156113.
  32. Nonparametric modeling and prognosis of condition monitoring signals using multivariate gaussian convolution processes. Technometrics, 60(4):484–496.
  33. A high-throughput multiparameter screen for accelerated development and optimization of soluble genetically encoded fluorescent biosensors. Nature Communications, 13(1):2919.
  34. Kushner, H. J. (1964). A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise.
  35. On-the-fly closed-loop materials discovery via bayesian active learning. Nature communications, 11(1):5966.
  36. Cokriging-based sequential design strategies using fast cross-validation techniques for multi-fidelity computer codes. Technometrics, 57(3):418–427.
  37. Uniform error bounds for gaussian process regression with application to safe control. Advances in Neural Information Processing Systems, 32.
  38. Efficient rollout strategies for bayesian optimization. In Conference on Uncertainty in Artificial Intelligence, pages 260–269. PMLR.
  39. Sequential bayesian design for accelerated life tests. Technometrics, 60(4):472–483.
  40. Deep gaussian processes for calibration of computer models (with discussion). Bayesian Analysis, 17(4):1301–1350.
  41. Inequalities: theory of majorization and its applications.
  42. Deep gaussian process emulation using stochastic imputation. Technometrics, pages 1–12.
  43. Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automatic Control, 54(1):48–61.
  44. Bayesian design of experiments using approximate coordinate exchange. Technometrics, 59(4):458–470.
  45. A novel approach for the optimal design of a biosensor. Analytical Letters, 53(9):1428–1445.
  46. Predicting sequential design decisions using the function-behavior-structure design process model and recurrent neural networks. Journal of Mechanical Design, 143(8).
  47. A stochastic approximation method. The annals of mathematical statistics, pages 400–407.
  48. Design and analysis of computer experiments. Statistical science, 4(4):409–423.
  49. Statistical designs and integral approximation. Proc. 12th Bienn. Semin. Can. Math. Congr.(R. Pyke, ed.), pages 115–136.
  50. Active learning for deep gaussian process surrogates. Technometrics, pages 1–15.
  51. Optimization of microfluidic biosensor efficiency by means of fluid flow engineering. Scientific Reports, 7(1):5721.
  52. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175.
  53. Distributed parametric consensus optimization with an application to model predictive consensus problem. IEEE Transactions on Cybernetics, 48(7):2024–2035.
  54. Scalable bayesian optimization using deep neural networks. In International conference on machine learning, pages 2171–2180. PMLR.
  55. Bayesian optimization with robust bayesian neural networks. Advances in neural information processing systems, 29.
  56. Making it stick: convection, reaction and diffusion in surface-based biosensors. Nature biotechnology, 26(4):417–426.
  57. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995.
  58. Sequential design of multi-fidelity computer experiments: maximizing the rate of stepwise uncertainty reduction. Technometrics, 64(2):199–209.
  59. Virtual library of simulation experiments: test functions and datasets. Simon Fraser University, Burnaby, BC, Canada, accessed May, 13:2015.
  60. Active emulation of computer codes with gaussian processes–application to remote sensing. Pattern Recognition, 100:107103.
  61. Electrochemical biosensors: recommended definitions and classification. Analytical Letters, 34(5):635–659.
  62. Bayesian optimization for chemical products and functional materials. Current Opinion in Chemical Engineering, 36:100728.
  63. On prediction properties of kriging: Uniform error bounds and robustness. Journal of the American Statistical Association, 115(530):920–930.
  64. The parallel knowledge gradient method for batch bayesian optimization. Advances in neural information processing systems, 29.
  65. Why non-myopic bayesian optimization is promising and how far should we look-ahead? a study via rollout. In International Conference on Artificial Intelligence and Statistics, pages 2808–2818. PMLR.
  66. Rapid, autonomous high-throughput characterization of hydrogel rheological properties via automated sensing and physics-guided machine learning. Applied Materials Today, 30:101720.
  67. Reduction of biosensor false responses and time delay using dynamic response and theory-guided machine learning. ACS sensors, 8(11):4079–4090.
  68. Improving biosensor accuracy and speed using dynamic signal change and theory-guided deep learning. Biosensors and Bioelectronics, 246:115829.
  69. Bayesian optimization for materials design with mixed quantitative and qualitative variables. Scientific reports, 10(1):1–13.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube