Papers
Topics
Authors
Recent
Search
2000 character limit reached

Inconsistencies in TeX-Produced Documents

Published 22 Jul 2024 in cs.SE | (2407.15511v2)

Abstract: TeX is a widely-used typesetting system adopted by most publishers and professional societies. While TeX is responsible for generating a significant number of documents, irregularities in the TeX ecosystem may produce inconsistent documents. These inconsistencies may occur across different TeX engines or different versions of TeX distributions, resulting in failures to adhere to formatting specifications, or the same document rendering differently for different authors. In this work, we investigate and quantify the robustness of the TeX ecosystem through a large-scale study of 432 documents. We developed an automated pipeline to evaluate the cross-engine and cross-version compatibility of the TeX ecosystem. We found significant inconsistencies in the outputs of different TeX engines: only 0.2% of documents compiled to identical output with XeTeX and PDFTeX due to a lack of cross-engine support in popular LaTeX packages and classes used in academic conferences. A smaller$\unicode{x2014}$yet significant$\unicode{x2014}$extent of inconsistencies was found across different TeX Live distributions, with only 42.1% of documents producing the same output from 2020 to 2023. Our automated pipeline additionally reduces the human effort in bug-finding: from a sample of 10 unique root causes of inconsistencies, we identified two new bugs in LaTeX packages and five existing bugs that were fixed independently of this study. We also observed potentially unintended inconsistencies across different TeX Live distributions beyond the updates listed in changelogs. We expect that this study will help authors of TeX documents to avoid unexpected outcomes by understanding how they may be affected by the often undocumented subtleties of the TeX ecosystem, while benefiting developers by demonstrating how different implementations result in unintended inconsistencies.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. TeXFAQ 2018. What are XeTeX and LuaTeX? TeXFAQ. https://texfaq.org/FAQ-xetex-luatex
  2. Overleaf 2022. Starting a New Era with 11 Million Users and 100 Million Projects. Overleaf. https://www.overleaf.com/blog/starting-a-new-era-with-11-million-users-and-100-million-projects
  3. arXiv 2023. Category Taxonomy. arXiv. https://arxiv.org/category_taxonomy
  4. TeX Users Group 2023. History of TeX. TeX Users Group. https://www.tug.org/whatis.html
  5. latex3 2023. ignoring space after \eqno #1059. latex3. https://github.com/latex3/latex2e/issues/1059
  6. The Royal Astronomical Society 2023. Instructions to Authors. The Royal Astronomical Society. https://academic.oup.com/mnras/pages/general_instructions
  7. The LaTeX Project 2023. LaTeX2e News Issue 38. The LaTeX Project. https://www.latex-project.org/news/latex2e-news/ltnews38.pdf
  8. Institute of Electrical and Electronics Engineers 2023. Manuscript Templates for Conference Proceedings. Institute of Electrical and Electronics Engineers. https://www.ieee.org/conferences/publishing/templates.html
  9. American Physical Society 2023. REVTeX Home Page. American Physical Society. https://journals.aps.org/revtex
  10. TeX Users Group 2023. What are TEX and its friends? TeX Users Group. https://www.ctan.org/tex
  11. Cornell University 2024. arXiv.org e-Print archive. Cornell University. https://arxiv.org/
  12. 2024. Setting curr file breaks compilation with images. https://www.dickimaw-books.com/bugtracker.php?key=273.
  13. arXiv 2024. TeX Submissions. arXiv. https://info.arxiv.org/help/submit_tex.html
  14. arXiv 2024. TeX::AutoTeX - automated processing of (La-)TeX sources - metacpan.org. arXiv. https://metacpan.org/pod/TeX::AutoTeX#SYNOPSIS
  15. TeX Users Group 2024a. TeXLive Availability. TeX Users Group. https://www.tug.org/texlive/acquire.html
  16. TeXlive 2024c. texlive/texlive - Docker Image — Docker Hub. TeXlive. https://hub.docker.com/r/texlive/texlive
  17. TeX Users Group 2024b. Welcome to TeX Live. TeX Users Group. https://www.tug.org/texlive//Contents/live/readme-html.dir/readme.en.html
  18. Why do developers use trivial packages? an empirical case study on npm. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 385–395. https://doi.org/10.1145/3106237.3106267
  19. Ivan Arraut. 2023. Gauge symmetries and the Higgs mechanism in Quantum Finance. Europhysics Letters 143, 4 (Aug. 2023), 42001. https://doi.org/10.1209/0295-5075/acedce
  20. Haim Bar and HaiYing Wang. 2021. Reproducible Science with LATEX. Journal of Data Science (2021), 111–125. https://doi.org/10.6339/21-jds998
  21. Paschalis Bizopoulos. 2023. A Makefile for Developing Containerized LaTeX Technical Documents. arXiv:2005.12660 [cs.SE]
  22. SciKGTeX – A LaTeX Package to Semantically Annotate Contributions in Scientific Publications. arXiv:2304.05327 [cs.DL]
  23. On the Reliability of Coverage-Based Fuzzer Benchmarking. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’22). Association for Computing Machinery, New York, NY, USA, 1621–1633. https://doi.org/10.1145/3510003.3510230
  24. Joppe W. Bos and Kevin S. McCurley. 2023. LaTeX, metadata, and publishing workflows. arXiv:2301.08277 [cs.DL]
  25. François Brischoux and Pierre Legagneux. 2009. Don’t Format Manuscripts. Scientist (Philadelphia, Pa.) 23 (04 2009), 24–24.
  26. Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement. Sociological Methods & Research 42, 3 (2013), 294–320. https://doi.org/10.1177/0049124113500475 arXiv:https://doi.org/10.1177/0049124113500475
  27. Coverage-directed differential testing of JVM implementations. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (Santa Barbara, CA, USA) (PLDI ’16). Association for Computing Machinery, New York, NY, USA, 85–99. https://doi.org/10.1145/2908080.2908095
  28. Yuting Chen and Zhendong Su. 2015. Guided differential testing of certificate validation in SSL/TLS implementations. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 793–804. https://doi.org/10.1145/2786805.2786835
  29. Understanding Integer Overflow in C/C++. ACM Trans. Softw. Eng. Methodol. 25, 1, Article 2 (dec 2015), 29 pages. https://doi.org/10.1145/2743019
  30. Automated Testing of Graphics Shader Compilers. Proc. ACM Program. Lang. 1, OOPSLA, Article 93 (oct 2017), 29 pages. https://doi.org/10.1145/3133917
  31. Symmetry-Aware Robot Design with Structured Subgroups. arXiv:2306.00036 [cs.AI]
  32. The pdfTeX user manual. Free Software Foundation. https://texdoc.org/serve/pdftex-a.pdf/0
  33. GrayC: Greybox Fuzzing of Compilers and Analysers for C. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (Seattle, WA, USA) (ISSTA 2023). Association for Computing Machinery, New York, NY, USA, 1219–1231. https://doi.org/10.1145/3597926.3598130
  34. FrankMittelbach. 2021. \\variants in tables are not robust. https://github.com/latex3/latex2e/issues/548.
  35. Ideograms for Physics and Chemistry. Foundations of Physics 46, 12 (Nov. 2016), 1713–1721. https://doi.org/10.1007/s10701-016-0044-5
  36. Alex Gaudeul. 2007. Do Open Source Developers Respond to Competition? The LATEX Case Study. Review of Network Economics 6, 2 (2007). https://doi.org/doi:10.2202/1446-9022.1119
  37. Differential regression testing for REST APIs. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual Event, USA) (ISSTA 2020). Association for Computing Machinery, New York, NY, USA, 312–323. https://doi.org/10.1145/3395363.3397374
  38. SAGE: whitebox fuzzing for security testing. Commun. ACM 55, 3 (mar 2012), 40–44. https://doi.org/10.1145/2093548.2093564
  39. LuaTeX Reference Manual. LuaTeX development team. http://mirrors.ibiblio.org/CTAN/systems/doc/luatex/luatex.pdf
  40. Revealing Performance Issues in Server-side WebAssembly Runtimes via Differential Testing. arXiv:2309.12167 [cs.SE]
  41. Yuma Jitsunari and Yoshitaka Arahori. 2019. Coverage-Guided Learning-Assisted Grammar-Based Fuzzing. In 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 275–280. https://doi.org/10.1109/ICSTW.2019.00065
  42. jovyntls. 2024a. Changes in horizontal spacing in some equations. https://github.com/latex3/latex2e/issues/1319.
  43. jovyntls. 2024b. Importing colortbl changes horizontal spacing. https://github.com/davidcarlisle/dpctex/issues/48.
  44. Tables to LaTeX: structure and content extraction from scientific tables. International Journal on Document Analysis and Recognition (IJDAR) 26, 2 (Oct. 2022), 121–130. https://doi.org/10.1007/s10032-022-00420-9
  45. Jonathan Kew. 2005. XeTeX, the Multilingual Lion: TEX meets Unicode and smart font technologies. In TUG 2005 Proceedings (Wuhan, China) (2005 International Typesetting Conference). TUGBoat, Portland, Oregon, U.S.A, 115–124.
  46. Finding Semantic Bugs in File Systems with an Extensible Fuzzing Framework. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (Huntsville, Ontario, Canada) (SOSP ’19). Association for Computing Machinery, New York, NY, USA, 147–161. https://doi.org/10.1145/3341301.3359662
  47. The Influence of Variable Frame Timing on First-Person Gaming. arXiv:2306.01691 [cs.GR]
  48. Differentially Testing Soundness and Precision of Program Analyzers. arXiv:1812.05033 [cs.SE]
  49. On the correctness of electronic documents: studying, finding, and localizing inconsistency bugs in PDF readers and files. Empir. Softw. Eng. 23, 6 (Dec. 2018), 3187–3220.
  50. Can You Accept LaTeX Files from Strangers? Ten Years Later. arXiv:2102.00856 [cs.CR]
  51. Program Reconditioning: Avoiding Undefined Behaviour When Finding and Reducing Compiler Bugs. Proc. ACM Program. Lang. 7, PLDI, Article 180 (jun 2023), 25 pages. https://doi.org/10.1145/3591294
  52. Shuxin Li and Manuel Rigger. 2024. Finding XPath Bugs in XML Document Processors via Differential Testing. arXiv:2401.05112 [cs.SE]
  53. Fuzzing: State of the Art. IEEE Transactions on Reliability 67, 3 (2018), 1199–1218. https://doi.org/10.1109/TR.2018.2834476
  54. Many-core compiler fuzzing. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (Portland, OR, USA) (PLDI ’15). Association for Computing Machinery, New York, NY, USA, 65–76. https://doi.org/10.1145/2737924.2737986
  55. GDsmith: Detecting Bugs in Graph Database Engines. arXiv:2206.08530 [cs.DB]
  56. T. Malathi and Manas Bhuyan. 2017. Performance analysis of Gabor wavelet for extracting most informative and efficient features. Multimedia Tools and Applications 76 (03 2017). https://doi.org/10.1007/s11042-016-3414-2
  57. Visualizing 2x2 Normal-Form Games: twoxtwogame LaTeX Package. arXiv:2402.16985 [cs.GT]
  58. William M. McKeeman. 1998. Differential Testing for Software. Digit. Tech. J. 10, 1 (1998), 100–107. https://www.hpl.hp.com/hpjournal/dtj/vol10num1/vol10num1art9.pdf
  59. Fuzz Revisited: A Re-Examination of the Reliability of UNIX Utilities and Services. (01 1998).
  60. An Empirical Study of the Robustness of MacOS Applications Using Random Testing. In Proceedings of the 1st International Workshop on Random Testing (Portland, Maine) (RT ’06). Association for Computing Machinery, New York, NY, USA, 46–54. https://doi.org/10.1145/1145735.1145743
  61. Frank Mittelbach. 2024. array package exhibits a surprising spacing difference when math cells end in explicit spaces. https://github.com/latex3/latex2e/issues/1323.
  62. TinyissimoYOLO: A Quantized, Low-Memory Footprint, TinyML Object Detection Network for Low Power Microcontrollers. In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, Hangzhou, China. https://doi.org/10.1109/aicas57966.2023.10168657
  63. Thorsten Ohl. 1995. Drawing Feynman diagrams with FX340-1 and METAFONT. Computer Physics Communications 90, 2–3 (Oct. 1995), 340–354. https://doi.org/10.1016/0010-4655(95)90137-s
  64. JEST: N+1-version Differential Testing of Both JavaScript Engines and Specification. arXiv:2102.07498 [cs.SE]
  65. Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction. arXiv:2308.05820 [cs.CV]
  66. Francesco Rullani. 2006. Dragging developers towards the core. How the Free/Libre/Open Source Software community enhances developers’ contribution. LEM Papers Series 2006/22. Laboratory of Economics and Management (LEM), Sant’Anna School of Advanced Studies, Pisa, Italy. https://ideas.repec.org/p/ssa/lemwps/2006-22.html
  67. AddressSanitizer: A Fast Address Sanity Checker. In 2012 USENIX Annual Technical Conference (USENIX ATC 12). USENIX Association, Boston, MA, 309–318. https://www.usenix.org/conference/atc12/technical-sessions/presentation/serebryany
  68. Sumeet S. Singh. 2018. Teaching Machines to Code: Neural Markup Generation with Visual Attention. arXiv:1802.05415 [cs.LG]
  69. Cornucopia : A Framework for Feedback Guided Generation of Binaries. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE ’22). ACM. https://doi.org/10.1145/3551349.3561152
  70. Pawan Sinha and Richard Russell. 2011. Perceptually-based Comparison of Image Similarity Metrics. Perception 40 (11 2011), 1269–81. https://doi.org/10.1068/p7063
  71. American Physical Society. 2018. REVTeX 4.2 Author’s Guide. American Physical Society, Ridge, NY, USA.
  72. Data-Oriented Differential Testing of Object-Relational Mapping Systems. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 1535–1547. https://doi.org/10.1109/ICSE43902.2021.00137
  73. Evgeniy Stepanov and Konstantin Serebryany. 2015. MemorySanitizer: Fast detector of uninitialized memory use in C++. In 2015 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). 46–55. https://doi.org/10.1109/CGO.2015.7054186
  74. Finding and analyzing compiler warning defects. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE ’16). Association for Computing Machinery, New York, NY, USA, 203–213. https://doi.org/10.1145/2884781.2884879
  75. Jovyn Tan and Manuel Rigger. 2024. Artefact for ”Inconsistencies in TeX-Produced Documents”. https://doi.org/10.5281/zenodo.12669708
  76. Bezhentcev Roman Vadimovich. 2013. Software for creating pictures in the LaTeX environment. arXiv:1304.0600 [cs.GR]
  77. Domonkos F. Vamossy. 2023. Social Media Emotions and IPO Returns. arXiv:2306.12602 [q-fin.PR]
  78. Zelun Wang and Jyh-Charn Liu. 2019. Translating Math Formula Images to LaTeX Sequences Using Deep Neural Networks with Sequence-level Training. arXiv:1908.11415 [cs.LG]
  79. Tsaihsuan Yang. 2017. An Intelligent and Automated PDF Format Checker. (2017).
  80. Finding and Understanding Bugs in C Compilers. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (San Jose, California, USA) (PLDI ’11). Association for Computing Machinery, New York, NY, USA, 283–294. https://doi.org/10.1145/1993498.1993532
  81. A. Zeller and R. Hildebrandt. 2002. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering 28, 2 (2002), 183–200. https://doi.org/10.1109/32.988498
  82. Finding bugs in Gremlin-based graph database systems via Randomized differential testing. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (¡conf-loc¿, ¡city¿Virtual¡/city¿, ¡country¿South Korea¡/country¿, ¡/conf-loc¿) (ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 302–313. https://doi.org/10.1145/3533767.3534409
  83. Fengmin Zhu and Fei He. 2022. EqFix: Fixing LaTeX Equation Errors by Examples. arXiv:2107.00613 [cs.PL]

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up