Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised Cross-Domain Soft Sensor Modelling via Deep Physics-Inspired Particle Flow Bayes (2306.04919v4)

Published 8 Jun 2023 in cs.LG

Abstract: Data-driven soft sensors are essential for achieving accurate perception through reliable state inference. However, developing representative soft sensor models is challenged by issues such as missing labels, domain adaptability, and temporal coherence in data. To address these challenges, we propose a deep Particle Flow Bayes (DPFB) framework for cross-domain soft sensor modeling in the absence of target state labels. In particular, a sequential Bayes objective is first formulated to perform the maximum likelihood estimation underlying the cross-domain soft sensing problem. At the core of the framework, we incorporate a physics-inspired particle flow that optimizes the sequential Bayes objective to perform an exact Bayes update of the model extracted latent and hidden features. As a result, these contributions enable the proposed framework to learn a rich approximate posterior feature representation capable of characterizing complex cross-domain system dynamics and performing effective time series unsupervised domain adaptation (UDA). Finally, we validate the framework on a complex industrial multiphase flow process system with complex dynamics and multiple operating conditions. The results demonstrate that the DPFB framework achieves superior cross-domain soft sensing performance, outperforming state-of-the-art deep UDA and normalizing flow approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Z. Y. Ding, J. Y. Loo, S. G. Nurzaman, C. P. Tan, and V. M. Baskaran, “A zero-shot soft sensor modeling approach using adversarial learning for robustness against sensor fault,” IEEE Transactions on Industrial Informatics, vol. 19, no. 4, pp. 5891–5901, 2023.
  2. S. Yang, K. Yu, F. Cao, L. Liu, H. Wang, and J. Li, “Learning causal representations for robust domain adaptation,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 3, pp. 2750–2764, 2023.
  3. Y. Shi, X. Ying, and J. Yang, “Deep unsupervised domain adaptation with time series sensor data: A survey,” Sensors, vol. 22, no. 15, 2022.
  4. A. Rozantsev, M. Salzmann, and P. Fua, “Beyond sharing weights for deep domain adaptation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 4, pp. 801–814, 2019.
  5. W. M. Kouw and M. Loog, “A review of domain adaptation without target labels,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 3, pp. 766–785, 2021.
  6. G. Wilson and D. J. Cook, “A survey of unsupervised deep domain adaptation,” ACM Trans. Intell. Syst. Technol., vol. 11, no. 5, 2020.
  7. W. Deng, L. Zhao, G. Kuang, D. Hu, M. Pietikäinen, and L. Liu, “Deep ladder-suppression network for unsupervised domain adaptation,” IEEE Transactions on Cybernetics, vol. 52, no. 10, pp. 10 735–10 749, 2022.
  8. S. Yang, K. Yu, F. Cao, H. Wang, and X. Wu, “Dual-representation-based autoencoder for domain adaptation,” IEEE Transactions on Cybernetics, vol. 52, no. 8, pp. 7464–7477, 2022.
  9. L. Wen, L. Gao, and X. Li, “A new deep transfer learning based on sparse auto-encoder for fault diagnosis,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 1, pp. 136–144, 2019.
  10. B. Yang, M. Ye, Q. Tan, and P. C. Yuen, “Cross-domain missingness-aware time-series adaptation with similarity distillation in medical applications,” IEEE Transactions on Cybernetics, vol. 52, no. 5, pp. 3394–3407, 2022.
  11. P. R. de Oliveira da Costa, A. Akçay, Y. Zhang, and U. Kaymak, “Remaining useful lifetime prediction via deep domain adaptation,” Reliability Engineering & System Safety, vol. 195, p. 106682, 2020.
  12. Z. Chai and C. Zhao, “A fine-grained adversarial network method for cross-domain industrial fault diagnosis,” IEEE Transactions on Automation Science and Engineering, vol. 17, no. 3, pp. 1432–1442, 2020.
  13. Z. Chai, C. Zhao, and B. Huang, “Multisource-refined transfer network for industrial fault diagnosis under domain and category inconsistencies,” IEEE Transactions on Cybernetics, vol. 52, no. 9, pp. 9784–9796, 2022.
  14. Z. Chen, Y. Liao, J. Li, R. Huang, L. Xu, G. Jin, and W. Li, “A multi-source weighted deep transfer network for open-set fault diagnosis of rotary machinery,” IEEE Transactions on Cybernetics, vol. 53, no. 3, pp. 1982–1993, 2023.
  15. J. Chen, J. Wang, and C. W. de Silva, “Mutual variational inference: An indirect variational inference approach for unsupervised domain adaptation,” IEEE Transactions on Cybernetics, vol. 52, no. 11, pp. 11 491–11 503, 2022.
  16. F. Wu and X. Zhuang, “Unsupervised domain adaptation with variational approximation for cardiac segmentation,” IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3555–3567, 2021.
  17. X. Liu, B. Hu, L. Jin, X. Han, F. Xing, J. Ouyang, J. Lu, G. El Fakhri, and J. Woo, “Domain generalization under conditional and label shifts via variational bayesian inference,” in International Joint Conference on Artificial Intelligence (IJCAI), 2021, pp. 881–887.
  18. X. Liu, S. Li, Y. Ge, P. Ye, J. You, and J. Lu, “Ordinal unsupervised domain adaptation with recursively conditional gaussian imposed variational disentanglement,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–14, 2022.
  19. S. Purushotham, W. Carvalho, T. Nilanon, and Y. Liu, “Variational recurrent adversarial deep domain adaptation,” in International Conference on Learning Representations (ICLR), 2017.
  20. Y. Tu, M.-W. Mak, and J.-T. Chien, “Variational domain adversarial learning with mutual information maximization for speaker verification,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 2013–2024, 2020.
  21. Z. Chai, C. Zhao, B. Huang, and H. Chen, “A deep probabilistic transfer learning framework for soft sensor modeling with missing data,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 7598–7609, 2022.
  22. S. Sapai, J. Y. Loo, Z. Y. Ding, C. P. Tan, R. C. Phan, V. M. Baskaran, and S. G. Nurzaman, “Cross-domain transfer learning and state inference for soft robots via a semi-supervised sequential variational bayes framework,” in International Conference on Robotics and Automation (ICRA), 2023.
  23. D. Kingma and M. Welling, “Auto-encoding variational bayes,” in International Conference on Learning Representations (ICLR), 2013.
  24. R. Xie, N. M. Jan, K. Hao, L. Chen, and B. Huang, “Supervised variational autoencoders for soft sensor modeling with missing data,” IEEE Transactions on Industrial Informatics, vol. 16, no. 4, pp. 2820–2828, 2020.
  25. D. Rezende and S. Mohamed, “Variational inference with normalizing flows,” in International Conference on Machine Learning (ICML), vol. 37, 2015, pp. 1530–1538.
  26. X. Yuan, L. Li, and Y. Wang, “Nonlinear dynamic soft sensor modeling with supervised long short-term memory network,” IEEE Transactions on Industrial Informatics, vol. 16, no. 5, pp. 3168–3176, 2020.
  27. J. Zhang, L. Qi, Y. Shi, and Y. Gao, “Generalizable model-agnostic semantic segmentation via target-specific normalization,” Pattern Recognition, vol. 122, p. 108292, 2022.
  28. J. Chung, K. Kastner, L. Dinh, K. Goel, A. C. Courville, and Y. Bengio, “A recurrent latent variable model for sequential data,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 28, 2015.
  29. A. Doucet, A. M. Johansen et al., “A tutorial on particle filtering and smoothing: Fifteen years later,” Handbook of nonlinear filtering, vol. 12, no. 656-704, p. 3, 2009.
  30. F. Daum and J. Huang, “Nonlinear filters with log-homotopy,” in Signal and Data Processing of Small Targets, vol. 6699, 2007, p. 669918.
  31. D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling, “Improved variational inference with inverse autoregressive flow,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 29, 2016.
  32. S. Pal, L. Ma, Y. Zhang, and M. Coates, “Rnn with particle flow for probabilistic spatio-temporal forecasting,” in International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 139, 2021, pp. 8336–8348.
  33. S. C. Surace, A. Kutschireiter, and J.-P. Pfister, “How to avoid the curse of dimensionality: Scalability of particle filters with and without importance weights,” SIAM Review, vol. 61, no. 1, pp. 79–91, 2019.
  34. A. Vahdat and J. Kautz, “Nvae: A deep hierarchical variational autoencoder,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 19 667–19 679.
  35. T. Yang, H. A. P. Blom, and P. G. Mehta, “The continuous-discrete time feedback particle filter,” in American Control Conference (ACC), 2014, pp. 648–653.
  36. S. Y. Olmez, A. Taghvaei, and P. G. Mehta, “Deep fpf: Gain function approximation in high-dimensional setting,” in 2020 59th IEEE Conference on Decision and Control (CDC), 2020, pp. 4790–4795.
  37. W. M. Czarnecki, S. Osindero, M. Jaderberg, G. Swirszcz, and R. Pascanu, “Sobolev training for neural networks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.
  38. C. Ruiz-Cárcel, Y. Cao, D. Mba, L. Lao, and R. Samuel, “Statistical process monitoring of a multiphase flow facility,” Control Engineering Practice, vol. 42, pp. 74–88, 2015.
  39. I. Prémont-Schwarz, A. Ilin, T. Hao, A. Rasmus, R. Boney, and H. Valpola, “Recurrent ladder networks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.
  40. D. Qian and W. K. Cheung, “Learning hierarchical variational autoencoders with mutual information maximization for autoregressive sequence modeling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 2, pp. 1949–1962, 2023.
  41. J. Behrmann, W. Grathwohl, R. T. Q. Chen, D. Duvenaud, and J.-H. Jacobsen, “Invertible residual networks,” in International Conference on Machine Learning (ICML), vol. 97, 2019, pp. 573–582.
  42. C. J. Maddison, J. Lawson, G. Tucker, N. Heess, M. Norouzi, A. Mnih, A. Doucet, and Y. Teh, “Filtering variational objectives,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now