Papers
Topics
Authors
Recent
2000 character limit reached

Auto311: A Confidence-guided Automated System for Non-emergency Calls (2312.14185v2)

Published 19 Dec 2023 in cs.CL, cs.AI, cs.CY, and cs.LG

Abstract: Emergency and non-emergency response systems are essential services provided by local governments and critical to protecting lives, the environment, and property. The effective handling of (non-)emergency calls is critical for public safety and well-being. By reducing the burden through non-emergency callers, residents in critical need of assistance through 911 will receive a fast and effective response. Collaborating with the Department of Emergency Communications (DEC) in Nashville, we analyzed 11,796 non-emergency call recordings and developed Auto311, the first automated system to handle 311 non-emergency calls, which (1) effectively and dynamically predicts ongoing non-emergency incident types to generate tailored case reports during the call; (2) itemizes essential information from dialogue contexts to complete the generated reports; and (3) strategically structures system-caller dialogues with optimized confidence. We used real-world data to evaluate the system's effectiveness and deployability. The experimental results indicate that the system effectively predicts incident type with an average F-1 score of 92.54%. Moreover, the system successfully itemizes critical information from relevant contexts to complete reports, evincing a 0.93 average consistency score compared to the ground truth. Additionally, emulations demonstrate that the system effectively decreases conversation turns as the utterance size gets more extensive and categorizes the ongoing call with 94.49% mean accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Afonso, W. 2021. Planning for the unknown: Local government strategies from the fiscal year 2021 budget season in response to the COVID-19 pandemic. State and Local Government Review, 53(2): 159–171.
  2. A survey of text classification algorithms. Mining text data, 163–222.
  3. Amazon Web Services. 2023. Amazon Lex. https://aws.amazon.com/lex/.
  4. Anthropic. 2023. Claude2. https://www.anthropic.com. Accessed: 2023-08-03.
  5. Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3293–3302.
  6. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  7. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
  8. NLTK: The Natural Language Toolkit. http://www.nltk.org/. Online; accessed [insert date].
  9. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3: 993–1022.
  10. Bredin, H. 2020. pyannote.speaker: Speaker Diarization. Accessed: 2023-08-03.
  11. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901.
  12. YAKE! Collection-Independent Automatic Keyword Extractor. In European Conference on Information Retrieval. Springer.
  13. Universal Sentence Encoder. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 169–174.
  14. Coordination in emergency response management. Communications of the ACM, 51(5): 66–73.
  15. Cityspec: An intelligent assistant system for requirement specification in smart cities. In 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 32–39. IEEE.
  16. CitySpec with shield: A secure intelligent assistant for requirement formalization. Pervasive and Mobile Computing, 92: 101802.
  17. An Intelligent Assistant for Converting City Requirements to Formal Specification. In 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 174–176. IEEE.
  18. mBART: multidimensional monotone BART. Bayesian Analysis, 17(2): 515–544.
  19. Damerau, F. J. 1964. A technique for computer detection and correction of spelling errors. Communications of the ACM, 7(3): 171–176.
  20. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  21. Core techniques of question answering systems over knowledge bases: a survey. Knowledge and Information systems, 55: 529–569.
  22. Face, H. 2022. BERT: Pre-trained Transformer for Question Answering. Accessed: 2023-08-03.
  23. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, 1050–1059. PMLR.
  24. Google. 2023. Dialogflow. https://cloud.google.com/dialogflow.
  25. CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review. arXiv preprint arXiv:2103.06268.
  26. Long short-term memory. Neural computation, 9(8): 1735–1780.
  27. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110: 457–506.
  28. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551.
  29. Kim, Y. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.
  30. Text classification algorithms: A survey. Information, 10(4): 150.
  31. Recurrent convolutional neural networks for text classification. In Twenty-ninth AAAI conference on artificial intelligence.
  32. Lin, C.-Y. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out, 74–81. Barcelona, Spain: Association for Computational Linguistics.
  33. A novel spatial–temporal specification-based monitoring system for smart cities. IEEE Internet of Things Journal, 8(15): 11793–11806.
  34. Data sets, modeling, and decision making in smart cities: A survey. ACM Transactions on Cyber-Physical Systems, 4(2): 1–28.
  35. Predictive monitoring with logic-calibrated uncertainty for cyber-physical systems. ACM Transactions on Embedded Computing Systems (TECS), 20(5s): 1–25.
  36. Cityresolver: a decision support system for conflict resolution in smart cities. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), 55–64. IEEE.
  37. Toward formal methods for smart cities. Computer, 54(9): 39–48.
  38. Communication challenges in emergency response. Communications of the ACM, 50(3): 51–53.
  39. Deep learning–based text classification: a comprehensive review. ACM computing surveys (CSUR), 54(3): 1–40.
  40. A recent overview of the state-of-the-art elements of text classification. Expert Systems with Applications, 106: 36–54.
  41. A review of incident prediction, resource allocation, and dispatch models for emergency management. Accident Analysis & Prevention, 165: 106501.
  42. NYC911. 2022. Next-Gen 911 on Target for 2024 Completion. https://www.nyc.gov/content/oti/pages/press-releases/next-gen-911-on-target-2024-completion. Accessed: 08-15-2023.
  43. OpenAI. 2022. Whisper: Speech Recognition. Accessed: 2023-08-03.
  44. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730–27744.
  45. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311–318.
  46. Learning from scratch a confidence measure. In Bmvc, volume 2, 4.
  47. Quantitative evaluation of confidence measures in a machine learning world. In Proceedings of the IEEE International Conference on Computer Vision, 5228–5237.
  48. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. In Advances in Neural Information Processing Systems (NeurIPS).
  49. Know what you don’t know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822.
  50. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  51. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084.
  52. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
  53. Annual Survey of Public Employment & Payroll Summary Report: 2021. US Census Bureau.
  54. Applications of artificial intelligence for disaster management. Natural Hazards, 103(3): 2631–2689.
  55. suno-ai. 2023. Text-Prompted Generative Audio Model. https://github.com/suno-ai/bark. Accessed: 9 August 2023.
  56. Attention is all you need. Advances in neural information processing systems, 30.
  57. Emergency response in natural disaster management: Allocation and scheduling of rescue units. European Journal of Operational Research, 235(3): 697–708.
  58. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. https://github.com/huggingface/transformers. Accessed: 2023-02-20.
  59. Big bird: Transformers for longer sequences. Advances in neural information processing systems, 33: 17283–17297.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up