DepMiner: Mining Intra-Project Dependencies

• DepMiner is a language-agnostic tool that extracts fine-grained intra-project dependencies using IntelliJ’s PSI, indexing, and reference resolution.
• It leverages headless execution and JSON output to integrate seamlessly into ETL pipelines for large-scale, automated mining workflows.
• By reusing mature IDE infrastructure, DepMiner enhances research reproducibility while providing precise and machine-readable dependency data.

Searching arXiv for DepMiner and closely related entries to ground the article in the relevant paper(s). arXiv search query: all:DepMiner arXiv search query: ti:"DepMiner" OR abs:"DepMiner" arXiv search query: all:"intra-project dependencies" AND all:IntelliJ DepMiner is an open-source, language-agnostic tool for mining intra-project source-code dependencies in a form suitable for downstream research analysis. Introduced in “DepMiner: A Pipelineable Tool for Mining of Intra-Project Dependencies” (Repinskiy et al., 2021), it occupies a specific niche within software-engineering tooling: rather than prioritizing interactive visualization, architecture dashboards, or coarse summary metrics, it exposes raw, machine-readable dependency data derived from source code and is designed to run headlessly inside larger mining pipelines. Its central design choice is to reuse the static analysis infrastructure of the IntelliJ Platform, especially PSI-based parsing, indexing, and reference resolution, instead of implementing language-specific front ends from scratch.

1. Research problem and intended role

DepMiner is motivated by a practical gap in empirical software engineering. Dependency analysis is central to software maintenance, quality control, performance and maintainability assessment, architecture monitoring, refactoring support, and the detection of architectural erosion and technical debt. In research, dependency graphs are used for defect prediction via network analysis, vulnerability prediction, software clustering, modularity analysis, architectural reconstruction, and program comprehension. The paper argues, however, that many existing tools are aimed primarily at developers and architects, are tied to particular languages, emphasize visualization or derived metrics, or do not expose raw dependency data in a format suitable for large-scale mining workflows (Repinskiy et al., 2021).

The problem addressed by DepMiner is therefore not dependency analysis in the abstract, but the difficulty of building reusable and scalable extraction pipelines for research. Extracting fine-grained intra-project dependency data typically requires substantial language-specific engineering for parsing, symbol indexing, and name resolution. DepMiner’s response is to repurpose infrastructure that integrated development environments already maintain for code intelligence. This makes the tool research-oriented in a specific sense: it is meant to lower the engineering cost of obtaining dependency data that can then be consumed by graph analysis, visualization, clustering, machine learning, or statistical workflows.

The paper positions the tool as “pipelineable.” In this context, pipelineability means command-line execution, headless operation, batch processing over multiple repositories, and JSON output as a standard interchange format. This suggests that DepMiner is best understood not as a standalone analysis environment, but as a dependency-extraction layer suitable for insertion into ETL-style repository-mining systems.

2. System architecture and extraction workflow

DepMiner is implemented as an IntelliJ IDEA plugin together with supporting infrastructure for running IntelliJ in headless mode. This architecture is central to the tool’s operation. Rather than building parsers, semantic models, and resolvers independently, the tool delegates those tasks to the IntelliJ Platform and its language-support plugins, then serializes the resulting dependency information for external consumption (Repinskiy et al., 2021).

The extraction pipeline described in the paper proceeds conceptually as follows: IntelliJ IDEA is started in headless mode, a target project is opened or created programmatically, indexing is triggered, PSI trees are built for files in scope, PSI elements are traversed, reference-bearing elements are identified via getReferences(), each reference is resolved with resolve(), and the resulting usage-to-declaration relations are written to JSON. The primary input is a source-code project directory. Projects with IDE-recognized metadata such as .project and .iml can be opened automatically with IntelliJ utilities, while projects without such metadata can be set up programmatically; the paper identifies ProjectSetupUtil.kt as the location of that setup logic.

This execution model gives DepMiner several properties that matter for research-scale automation. Headless execution avoids GUI interaction, supports bulk processing, and allows incorporation into command-line workflows. Because indexing, PSI construction, and resolution are provided by IntelliJ, the tool inherits industrial-strength language analysis infrastructure. The paper does not claim benchmark-leading scalability, but its argument for large-scale suitability is architectural: reuse of mature IDE infrastructure, on-demand PSI construction, scope restriction, and machine-readable serialization.

3. Dependency representation and output model

DepMiner’s output is a JSON file containing a list of dependency records. The paper does not reproduce a full schema verbatim, but it describes the data model clearly. Each dependency connects two code-element records, and each code-element record combines a location marker with a type signature (Repinskiy et al., 2021).

The location marker consists of an absolute path to the file containing the code element together with the range of lines that the element spans. The type signature is a language-specific description derived from a field of the corresponding PSI element; examples given in the paper include Java-specific labels such as PsiClass and PsiReferenceExpression. In conceptual form, the paper describes the representation as:

CodeElement = (LocationMarker, TypeSignature) Dependency = (CodeElement_source, CodeElement_target)

This representation is significant because it preserves both positional and semantic information. The explicit storage of spans and file paths allows downstream analyses to infer containment relationships, such as a method being inside a class or a class being inside a file. The paper notes that such location information is crucial for deriving higher-level inter-class views from lower-level dependencies. The JSON output is therefore intended to support multiple later transformations: graph construction, filtering by token/class/file/directory level, aggregation into architectural views, import into graph databases, and use in notebooks or visualization applications.

A notable aspect of the design is that the exported dependencies are fine-grained and reference-based. The tool is not limited to emitting only high-level module or file relations. Instead, it serializes low-level usage-to-declaration information from which coarser dependency structures can later be aggregated.

4. Static analysis substrate: PSI, indexing, and reference resolution

Technically, DepMiner relies on three IntelliJ Platform mechanisms: the Program Structure Interface (PSI), IntelliJ indexing, and IntelliJ reference resolution. PSI is described in the paper as an AST-like but richer semantic representation intended for IDE functionality. Nodes correspond to meaningful program elements, with Java examples including PsiClass, PsiMethod, and PsiField; node types carry metadata such as location, containing file, language, and element-specific properties (Repinskiy et al., 2021).

PSI trees are built on demand rather than eagerly for the entire project. This matters because it limits unnecessary work during batch mining. When a project is opened, IntelliJ indexes the source tree by storing keys and associated values in binary form for efficient querying. If a PSI element represents a usage of another element, it can provide one or more PSI references via getReferences(). A reference does not directly store the declaration location; rather, it stores logic for locating the declaration. Calling resolve() initiates a lookup process that uses the indexes to find candidate declarations and then checks whether they match.

The paper’s method can therefore be summarized as reference-based dependency mining. DepMiner traverses PSI trees, inspects each element for references, resolves those references, and emits a dependency object for each resolved usage-to-declaration relation. This formulation also clarifies the scope of what the tool extracts. The paper is slightly inconsistent in places, once describing DepMiner as a tool for call-graph extraction, but its implementation description is broader: it extracts dependencies by resolving PSI references in general, not method calls alone. The examples include class usage, method-call references, and variable or token-level references. This suggests that the most accurate characterization is that DepMiner mines reference-based intra-project dependencies whose exact variety depends on what the relevant language-support plugin exposes through PSI.

5. Language support, granularity, and extensibility

DepMiner is described as language-agnostic, but the paper gives that term a specific implementation meaning. It does not mean the existence of a universal parser or language-independent dependency semantics. Rather, the tool’s core logic is written against IntelliJ Platform abstractions such as PSI and references, while language-specific parsing and PSI construction are delegated to IntelliJ language-support plugins. The paper states that the initial language support requirement covered Java and Python, and that support for additional languages can be added by depending on the corresponding IntelliJ plugin in build.gradle.kts (Repinskiy et al., 2021).

The same architectural choice underlies DepMiner’s granularity claims. The requirements section states that the tool must extract dependencies at all levels of granularity from project source code: tokens, functions and classes encapsulating them, files, and directories. In practice, this is achieved indirectly. The extracted data are rooted in fine-grained PSI elements with spans and file paths, and higher-level relations can then be inferred by containment and aggregation. This makes the output adaptable to different research questions without requiring the extractor itself to emit separate schemas for each granularity.

The paper describes three extensibility directions. First, new languages can be supported by adding the relevant IntelliJ plugin dependency. Second, the scope of analysis can be customized via an AnalysisScope interface, allowing exclusion of particular files, directories, or line ranges; the paper presents this both as a customization mechanism and as a performance optimization. Third, the same plugin architecture can be repurposed for mining information other than dependencies, because it provides access to any information available within the IDE. This suggests that DepMiner is framed not only as a finished extractor, but also as a reusable basis for other PSI-centric source-mining tools.

A common misconception is that “language-agnostic” here implies uniform semantics across languages. The paper explicitly supports a narrower interpretation: framework-level agnosticism through a common IDE abstraction layer. Language breadth and analysis quality therefore remain contingent on the availability and maturity of IntelliJ language-support plugins.

6. Evaluation status, comparative positioning, and limitations

DepMiner is positioned against several classes of existing tools. The paper distinguishes it from visualization and architecture tools such as Structure101, MagicDraw, Sourcetrail, and OptimalJ; from metrics-oriented tools such as JDepend, JavaNCSS, Dependency Finder, and CppDepend; and from vulnerability tools such as OSS Index, OWASP Dependency-Check, and RetireJS. It also notes that IntelliJ itself can analyze project dependencies at module, file, and line levels, but does not conveniently export that data for multi-project mining runs. DepMiner’s differentiator is therefore not a novel dependency-analysis algorithm, but the packaging of IDE static analysis into a reusable mining backend with headless execution and JSON output (Repinskiy et al., 2021).

The paper does not present a conventional empirical evaluation. There are no benchmark datasets, no extraction-accuracy measurements, no precision or recall tables, and no comparative runtime experiments against alternative dependency miners. Instead, it provides a demonstration artifact: a web application that visualizes the dependency graph produced by DepMiner. The performance discussion is qualitative. The authors state that constructing and analyzing large PSI trees can consume substantial memory, that resolving and searching for references is computationally expensive, and that the analysis scope should be restricted when possible to avoid pipeline bottlenecks. Essential functions are said to be covered by functional tests.

These omissions define the main limitations of the work. Performance overhead is acknowledged rather than quantified. Support breadth depends on IntelliJ plugins. Dependency-type coverage is under-specified, even though the implementation evidently goes beyond call graphs. The tool is explicitly framed for intra-project dependencies, not inter-project ecosystems or package-manager analysis. Higher-level aggregation procedures are left to downstream processing. This suggests that the paper’s strongest contribution is infrastructural rather than evaluative: it provides a plausible and technically coherent research tool, but not a comprehensive empirical characterization of correctness, completeness, or scalability.

In sum, DepMiner is a research-oriented dependency extractor that turns IntelliJ’s PSI-based static analysis into a pipelineable source-mining service. Its importance lies in its operationalization of intra-project dependency mining for empirical studies: command-line execution, headless processing, fine-grained reference resolution, JSON serialization, extensibility across languages and scopes, and preservation of enough location and type metadata to support subsequent graph and architectural analysis (Repinskiy et al., 2021).