Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 91 tok/s
Gemini 2.5 Pro 47 tok/s Pro
GPT-5 Medium 30 tok/s
GPT-5 High 33 tok/s Pro
GPT-4o 98 tok/s
GPT OSS 120B 483 tok/s Pro
Kimi K2 242 tok/s Pro
2000 character limit reached

PLMM: Personal Large Language Models on Mobile Devices (2309.14726v2)

Published 26 Sep 2023 in cs.CV, cs.AI, cs.CE, cs.CL, and cs.LG

Abstract: Inspired by Federated Learning, in this paper, we propose personal large models that are distilled from traditional LLMs but more adaptive to local users' personal information such as education background and hobbies. We classify the LLMs into three levels: the personal level, expert level and traditional level. The personal level models are adaptive to users' personal information. They encrypt the users' input and protect their privacy. The expert level models focus on merging specific knowledge such as finance, IT and art. The traditional models focus on the universal knowledge discovery and upgrading the expert models. In such classifications, the personal models directly interact with the user. For the whole system, the personal models have users' (encrypted) personal information. Moreover, such models must be small enough to be performed on personal computers or mobile devices. Finally, they also have to response in real-time for better user experience and produce high quality results. The proposed personal large models can be applied in a wide range of applications such as language and vision tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” CoRR, vol. abs/2005.14165, 2020. [Online]. Available: https://arxiv.org/abs/2005.14165
  2. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, no. 1, jan 2020.
  3. J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds.   Association for Computational Linguistics, 2019, pp. 4171–4186.
  4. N. Chenouard, I. Smal, F. de Chaumont, M. Maska, I. F. Sbalzarini, Y. Gong, J. Cardinale, C. Carthel, S. Coraluppi, M. Winter, A. R. Cohen, W. J. Godinez, K. Rohr, Y. Kalaidzidis, L. Liang, J. Duncan, H. Shen, Y. Xu, K. E. G. Magnusson, J. Jalden, H. M. Blau, P. Paul-Gilloteaux, P. Roudot, C. Kervrann, F. Waharte, J.-Y. Tinevez, S. L. Shorte, J. Willemse, K. Celler, G. P. van Wezel, H.-W. Dan, Y.-S. Tsai, C. Ortiz de Solorzano, J.-C. Olivo-Marin, and E. Meijering, “Objective comparison of particle tracking methods,” Nat. Methods, vol. 11, no. 3, pp. 281–U247, March 2014.
  5. Y. Gong, Q. Wang, C. Yang, Y. Gao, and C. Li, “Symmetry detection for multi-object using local polar coordinate,” Lecture Notes in Computer Science, vol. 5702, p. 277, 2009.
  6. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” CoRR, vol. abs/1910.13461, 2019. [Online]. Available: http://arxiv.org/abs/1910.13461
  7. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, Y. Du, C. Yang, Y. Chen, Z. Chen, J. Jiang, R. Ren, Y. Li, X. Tang, Z. Liu, P. Liu, J.-Y. Nie, and J.-R. Wen, “A survey of large language models,” 2023.
  8. Y. Gong, G. Paul, and I. F. Sbalzarini, “Coupled signed-distance functions for implicit surface reconstruction,” in IEEE Intl. Symp. Biomed. Imaging (ISBI), May 2012, pp. 1000–1003.
  9. Y. Gong and I. F. Sbalzarini, “Local weighted Gaussian curvature for image processing,” Intl. Conf. Image Proc. (ICIP), pp. 534–538, September 2013.
  10. L. Yu and M. T. Orchard, “Single image interpolation exploiting semi-local similarity.”   Brighton, UK: IEEE, 2019, pp. 1722–1726.
  11. Y. Gong and I. F. Sbalzarini, “Image enhancement by gradient distribution specification,” in In Proc. Workshop ”Emerging Topics in Image Enhancement and Restoration”, 12th Asian Conference on Computer Vision, Singapore, Nov 2014, pp. w7–p3.
  12. H. Yin, Y. Gong, and G. Qiu, “Side window filtering,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 8750–8758.
  13. Y. Gong, “Spectrally regularized surfaces,” Ph.D. dissertation, ETH Zurich, Nr. 22616, 2015, http://dx.doi.org/10.3929/ethz-a-010438292.
  14. L. Yu, D. Liu, H. Mansour, and P. T. Boufounos, “Fast and high-quality blind multi-spectral image pansharpening,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–17, 2022.
  15. Y. Gong and I. Sbalzarini, “A natural-scene gradient distribution prior and its application in light-microscopy image processing,” Selected Topics in Signal Processing, IEEE Journal of, vol. 10, no. 1, pp. 99–114, Feb 2016.
  16. H. Guo and X. Yu, “A survey on blockchain technology and its security,” Blockchain: Research and Applications, vol. 3, no. 2, p. 100067, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2096720922000070
  17. Y. Gong and I. F. Sbalzarini, “Curvature filters efficiently reduce certain variational energies,” IEEE Transactions on Image Processing, vol. 26, no. 4, pp. 1786–1798, April 2017.
  18. M. Zong, R. Wang, X. Chen, Z. Chen, and Y. Gong, “Motion saliency based multi-stream multiplier resnets for action recognition,” Image and Vision Computing, vol. 107, p. 104108, 2021.
  19. Y. Gong, “Bernstein filter: A new solver for mean curvature regularized models,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2016, pp. 1701–1705.
  20. Y. Ezawa, S. Kakei, Y. Shiraishi, M. Mohri, and M. Morii, “Blockchain-based cross-domain authorization system for user-centric resource sharing,” Blockchain: Research and Applications, vol. 4, no. 2, p. 100126, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2096720923000015
  21. Y. Gong and Y. Xie, “Linear approximation of mean curvature,” in Image Processing (ICIP), 2017 IEEE International Conference on.   IEEE, 2017, pp. 570–574.
  22. W. Tang, L. Zhou, and Y. Gong, “Real-time optimizing weighted gaussian curvature for 4k videos,” in 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), 2021, pp. 1–6.
  23. Y. Gong, B. Liu, X. Hou, and G. Qiu, “Sub-window box filter,” in Proc. IEEE Visual Communications and Image Processing (VCIP), Dec. 2018, pp. 1–4.
  24. Y. Gong, X. Hou, F. Li, and G. Qiu, “Image filtering with generic geometric prior,” IEEE Access, vol. 6, pp. 54 320–54 330, 2018.
  25. L. Yu, D. Liu, H. Mansour, P. T. Boufounos, and Y. Ma, “Blind multi-spectral image pan-sharpening,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 1429–1433.
  26. Y. Gong and O. Goksel, “Weighted mean curvature,” Signal Processing, vol. 164, pp. 329 – 339, 2019.
  27. A. Sancheti and R. Rudinger, “What do large language models learn about scripts?” 2022.
  28. Y. Gong and Y. Chen, “Computing gaussian curvature in real-time for 4k video processing,” IEEE Access, vol. 7, pp. 115 936–115 944, 2019.
  29. W. Tang, Y. Gong, L. Su, W. Wu, and G. Qiu, “Structure adaptive filtering for edge-preserving image smoothing,” in Image and Graphics, Y. Peng, S.-M. Hu, M. Gabbouj, K. Zhou, M. Elad, and K. Xu, Eds.   Cham: Springer International Publishing, 2021, pp. 265–276.
  30. Y. Gong, H. Yin, J. Liu, B. Liu, and G. Qiu, “Soft tissue removal in x-ray images by half window dark channel prior,” in Proc. IEEE Int. Conf. Image Processing (ICIP), Sep. 2019, pp. 3576–3580.
  31. H. Yin, Y. Gong, and G. Qiu, “Side window guided filtering,” Signal Process., vol. 165, pp. 315–330, 2019.
  32. Y. Gong, “Computing curvature, mean curvature and weighted mean curvature.”   Bordeaux, France: IEEE, 2022, pp. 266–270.
  33. H. Yin, Y. Gong, and G. Qiu, “Fast and efficient implementation of image filtering using a side window convolutional neural network,” Signal Process., vol. 176, p. 107717, 2020.
  34. Y. Gong and Y. Chen, “Molecular surface estimation by geometric coupled distance functions,” IEEE Access, vol. 8, pp. 176 263–176 273, 2020.
  35. C. Jin, S. Pang, X. Qi, Z. Zhang, and A. Zhou, “A high performance concurrency protocol for smart contracts of permissioned blockchain,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 11, pp. 5070–5083, 2022.
  36. Y. Gong, W. Tang, L. Zhou, L. Yu, and G. Qiu, “Quarter laplacian filter for edge aware image processing,” in Proc. IEEE Int. Conf. Image Processing (ICIP), 2021, pp. 1959–1963.
  37. W. Tang, L. Zhou, and Y. Gong, “Curvature-based real-time brightness adjustment for ultra hd video,” in 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), 2022, pp. 1–6.
  38. Y. Gong, W. Tang, L. Zhou, L. Yu, and G. Qiu, “A discrete scheme for computing image’s weighted gaussian curvature,” in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 1919–1923.
  39. W. Tang, Y. Gong, and G. Qiu, “A novel structure adaptive algorithm for feature-preserving 3d mesh denoising,” in 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), 2022, pp. 1–6.
  40. Y. Gong, W. Huang, and W. Wu, “Removing scattered light in biomedical images,” in 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 2023, pp. 1–5.
  41. W. Tang, Z. Lin, and Y. Gong, “Gc-net: An unsupervised network for gaussian curvature optimization on images,” Journal of Signal Processing Systems, vol. 95, no. 1, pp. 77–88, 2023. [Online]. Available: https://doi.org/10.1007/s11265-022-01800-4
  42. W. Tang, Y. Gong, and G. Qiu, “Feature preserving 3d mesh denoising with a dense local graph neural network,” vol. 233, p. 103710, 2023.
  43. Y. Gong, “Imposing total variation prior into guided filter,” in Proc. IEEE Int. Conf. Image Processing (ICIP), 2023, pp. 156–160.
  44. M. Xu, Z. Zhang, Y. Gong, and S. Poslad, “Regression-based camera pose estimation through multi-level local features and global features,” Sensors, vol. 23, no. 8, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23/8/4063
  45. Y. Gong, “A multiscale residual solver for total variation models,” in Proc. IEEE Int. Conf. Image Processing (ICIP), 2023, pp. 151–155.
  46. Y. Han, G. Huang, S. Song, L. Yang, H. Wang, and Y. Wang, “Dynamic neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 7436–7456, 2022.
  47. Z. Zhao, W. Wu, H. Liu, and Y. Gong, “A multi-stream network for mesh denoising via graph neural networks with gaussian curvature,” in 2023 IEEE International Conference on Image Processing (ICIP), 2023, pp. 1355–1359.
  48. J. Scheurer, J. A. Campos, T. Korbak, J. S. Chan, A. Chen, K. Cho, and E. Perez, “Training language models with language feedback at scale,” Mar. 2023.
  49. R. Zhang, J. Han, A. Zhou, X. Hu, S. Yan, P. Lu, H. Li, P. Gao, and Y. Qiao, “Llama-adapter: Efficient fine-tuning of language models with zero-init attention,” CoRR, vol. abs/2303.16199, 2023.
  50. J. Zhao, Z. Yu, X. Zhang, and Y. Yang, “Overcoming language priors via shuffling language bias for robust visual question answering,” IEEE Access, vol. 11, pp. 85 980–85 989, 2023.
  51. J. Yi, J. Tao, R. Fu, T. Wang, C. Y. Zhang, and C. Wang, “Adversarial multi-task learning for mandarin prosodic boundary prediction with multi-modal embeddings,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2963–2973, 2023.
  52. Y. Gong, Z. Lin, and J. Sun, “A lossless compression method for event cameras via removing spatial redundancy,” in 2023 9th International Conference on Computer and Communications (ICCC), 2023, pp. 1962–1966.
  53. ——, “Tssr: A truncated and signed square root activation function for neural networks,” in 2023 9th International Conference on Computer and Communications (ICCC), 2023, pp. 1978–1982.
  54. Y. Gong, M. Xu, Y. Li, and M. Magno, “Removing scattered light in biomedical images via an unsupervised deep neural network,” in 2023 IEEE EMBS Special Topic Conference on Data Science and Engineering in Healthcare, Medicine and Biology, 2023, pp. 65–66.
  55. Y. Gong, “Eggs: Edge guided gaussian splatting for radiance fields,” 2024.
  56. Y. Gong, L. Yu, and G. Yue, “Isotropic gaussian splatting for real-time radiance field rendering,” 2024.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (1)

  1. Yuanhao Gong

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube