Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
131 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding and Estimating Domain Complexity Across Domains (2312.13487v1)

Published 20 Dec 2023 in cs.AI

Abstract: AI systems, trained in controlled environments, often struggle in real-world complexities. We propose a general framework for estimating domain complexity across diverse environments, like open-world learning and real-world applications. This framework distinguishes between intrinsic complexity (inherent to the domain) and extrinsic complexity (dependent on the AI agent). By analyzing dimensionality, sparsity, and diversity within these categories, we offer a comprehensive view of domain challenges. This approach enables quantitative predictions of AI difficulty during environment transitions, avoids bias in novel situations, and helps navigate the vast search spaces of open-world domains.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Lessons in Play: An Introduction to Combinatorial Game Theory. CRC Press. URL: https://books.google.com/books?id=pfLRBQAAQBAJ.
  2. in: Searching for solutions in games and artificial intelligence, pp. 158–161.
  3. A model of inductive bias learning. CoRR abs/1106.0245. URL: http://arxiv.org/abs/1106.0245, arXiv:1106.0245.
  4. What is the average number of turns in a game of monopoly? URL: https://boardgames.stackexchange.com/questions/33181/what-is-the-average-number-of-turns-in-a-game-of-monopoly.
  5. Towards a unifying framework for formal theories of novelty. Proceedings of the AAAI Conference on Artificial Intelligence 35, 15047--15052. URL: https://ojs.aaai.org/index.php/AAAI/article/view/17766.
  6. Weibull-open-world (wow) multi-type novelty detection in cartpole3d. Algorithms 15. URL: https://www.mdpi.com/1999-4893/15/10/381, doi:10.3390/a15100381.
  7. Openai gym. arXiv:arXiv:1606.01540.
  8. Data science: Challenges and directions. CoRR abs/2006.16966. URL: https://arxiv.org/abs/2006.16966, arXiv:2006.16966.
  9. Dimension from covariance matrices. Chaos: An Interdisciplinary Journal of Nonlinear Science 27, 023101. URL: http://aip.scitation.org/doi/10.1063/1.4975063, doi:10.1063/1.4975063.
  10. Mastering monopoly -- the odds of winning. URL: https://www.onlinecasino.ca/monopoly-odds/.
  11. Monte-carlo tree search: A new framework for game ai, in: Proceedings of the Fourth Artificial Intelligence and Interactive Digital Entertainment Conference.
  12. A study on AI-based approaches for high-level decision making in highway autonomous driving, in: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3671--3676. doi:10.1109/SMC.2017.8123203.
  13. Toward defining a domain complexity measure across domains. AAAI Spring Symposium URL: https://usc-isi-i2.github.io/AAAI2022SS/papers/SSS-22_paper_79.pdf.
  14. Information-theoretic measures of dataset difficulty. CoRR abs/2110.08420. URL: https://arxiv.org/abs/2110.08420, arXiv:2110.08420.
  15. Minedojo: Building open-ended embodied agnets with internet-scale knowledge. arXiv:2206.08853 URL: https://arxiv.org/abs/2206.08853, arXiv:2206.08853.
  16. Chaos and Fractals: An Elementary Introduction. EBSCO ebook academic collection, OUP Oxford. URL: https://books.google.com/books?id=exnWM_ZHK0MC.
  17. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7, 179--188. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1469-1809.1936.tb02137.x, doi:https://doi.org/10.1111/j.1469-1809.1936.tb02137.x, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1469-1809.1936.tb02137.x.
  18. The Computational Beauty of Nature: Computer Explorations of Fractals, Chaos, Complex Systems, and Adaptation. A Bradford book, MIT Press. URL: https://www.amazon.com/dp/0262062003/?coliid=I6COO6CGLWZUA&colid=SQVB5SEZMI4L&psc=0&ref_=lv_ov_lig_dp_it.
  19. Polycraft world ai lab (pal): An extensible platform for evaluating artificial intelligence agents. arXiv preprint arXiv:2301.11891 .
  20. Probabilistic models of cognition: exploring representations and inductive biases. Trends in Cognitive Sciences 14, 357--364. URL: https://www.sciencedirect.com/science/article/pii/S1364661310001129, doi:https://doi.org/10.1016/j.tics.2010.05.004.
  21. Neurips 2019 competition: The minerl competition on sample efficient reinforcement learning using human priors. arXiv:1904.10079v3 URL: https://arxiv.org/abs/1904.10079, arXiv:1904.10079v3.
  22. Measuring universal intelligence: Towards an anytime intelligence test. Artificial Intelligence 174, 1508--1539. URL: https://www.sciencedirect.com/science/article/pii/S0004370210001554, doi:https://doi.org/10.1016/j.artint.2010.09.006.
  23. What do we need to build explainable ai systems for the medical domain? arXiv:1712.09923.
  24. Comparing measures of sparsity. IEEE Transactions on Information Theory 55, 4723--4741. doi:10.1109/TIT.2009.2027527.
  25. Robotics and artificial intelligence: a perspective on deliberation functions. AI Communications 27. doi:10.3233/AIC-130578.
  26. Computational assessment of sparsity in board games.
  27. How we understand “complexity” makes a difference: Lessons from critical systems thinking and the covid-19 pandemic in the uk. Systems 8. URL: https://www.mdpi.com/2079-8954/8/4/52, doi:10.3390/systems8040052.
  28. Domain mapping matrix-based metric for measuring system design complexity. IEEE Transactions on Engineering Management , 1--9doi:10.1109/TEM.2020.3004561.
  29. How Complexity Domain Impacts Software Development Process. URL: https://gamemakers.com/how-complexity-domain-impacts-software-development-process/.
  30. The cifar-10 dataset. URL: https://www.cs.toronto.edu/~kriz/cifar.html.
  31. Open-world learning for radically autonomous agents. Proceedings of the AAAI Conference on Artificial Intelligence 34, 13539--13543. URL: https://ojs.aaai.org/index.php/AAAI/article/view/7078, doi:10.1609/aaai.v34i09.7078.
  32. Data, power and bias in artificial intelligence. CoRR abs/2008.07341. URL: https://arxiv.org/abs/2008.07341, arXiv:2008.07341.
  33. Universal intelligence: A definition of machine intelligence. arXiv:0712.3329.
  34. Mathematics Stack Exchange, a. Determining the number of valid tictactoe board states in terms of board dimension. URL: https://math.stackexchange.com/q/613505, arXiv:https://math.stackexchange.com/q/613505. uRL:https://math.stackexchange.com/q/613505 (version: 2013-12-20).
  35. Mathematics Stack Exchange, b. Tictactoe state space choose calculation. URL: https://math.stackexchange.com/q/485752, arXiv:https://math.stackexchange.com/q/485752. uRL:https://math.stackexchange.com/q/485752 (version: 2018-05-23).
  36. in: The Need for Biases in Learning Generalizations.
  37. Visualizing mnist: An exploration of dimensionality reduction. URL: http://colah.github.io/posts/2014-10-Visualizing-MNIST/.
  38. Computational text analysis for social science: Model complexity and assumptions, in: Proc. of the NIPS Workshop on Comptuational Social Science and the Wisdom of Crowds.
  39. Measuring the complexity of domains used to evaluate ai systems. arXiv:2010.01985.
  40. Assaying the importance of system complexity for the systems engineering community. Systems Engineering 23, 579--596. URL: https://incose.onlinelibrary.wiley.com/doi/abs/10.1002/sys.21550, doi:https://doi.org/10.1002/sys.21550, arXiv:https://incose.onlinelibrary.wiley.com/doi/pdf/10.1002/sys.21550.
  41. Complexity measurements of the inter-domain management system design, in: Proceedings. Ninth IEEE International Conference on Networks, ICON 2001., pp. 2--7. doi:10.1109/ICON.2001.962308.
  42. Learning from domain complexity, in: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), European Language Resources Association (ELRA), Reykjavik, Iceland. pp. 2021--2028. URL: http://www.lrec-conf.org/proceedings/lrec2014/pdf/480_Paper.pdf.
  43. Science of artificial intelligence and learning for open-world novelty). SAIL-ON URL: https://iresearch-cms.tau.ac.il/sites/resauth.tau.ac.il/files/DARPA SAIL-ONHR001119S0038.pdf.
  44. A mathematical theory of communication. Bell System Technical Journal 27. URL: http://cs-www.cs.yale.edu/homes/yry/readings/general/shannon1948.pdf.
  45. Teaching science through video games. Nature Chemistry 9, 97--102. URL: https://doi.org/10.1038/nchem.2694, doi:10.1038/nchem.2694.
  46. Measuring Complexity of Domain Models Represented by Feature Diagrams. Information Technology And Control 38, 179--187.
  47. Wikipedia, . Game complexity --- Wikipedia, the free encyclopedia. URL: https://en.wikipedia.org/w/index.php?title=Game_complexity&oldid=1129116028. [Online; accessed 13-January-2023].
  48. Bounded expectations for discrepancy detection in goal-driven autonomy, in: AAAI Workshop - Technical Report, pp. 50--56.
  49. XboxPress, 2021. Minecraft franchise fact sheet. URL: https://news.xbox.com/en-us/wp-content/uploads/sites/2/2021/04/Minecraft-Franchise-Fact-Sheet_April-2021.pdf.
  50. An estimation method for game complexity. URL: https://arxiv.org/abs/1901.11161, doi:10.48550/ARXIV.1901.11161.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now