TechNet: Engineering Semantic Network

• TechNet is a large-scale, engineering semantic network built from U.S. patents that enables precise retrieval and mapping of technical concepts.
• It employs advanced NLP pipelines and SGNS Word2Vec embeddings to extract and represent over 4 million patent-based technical terms with 62 million semantic edges.
• TechNet underpins diverse applications such as prior-art search, design ideation, and innovation analysis by offering domain-specific accuracy and extensive coverage.

TechNet is a large-scale, engineering-centric semantic network whose nodes encode technical concepts—single words or multiword terms (up to four-grams)—extracted from the full U.S. utility patent corpus. Edges represent pairwise semantic relatedness as learned from the empirical co-occurrence patterns in patent text. TechNet’s principal objective is to support engineering knowledge representation, retrieval, and discovery with domain-specific accuracy and coverage, overcoming the limitations of general-purpose networks such as WordNet or ConceptNet. The network underpins diverse applications, including prior-art search, design ideation, technology forecasting, and the quantitative analysis of innovation and originality in the vast technological concept space (Sarica et al., 2020, Sarica et al., 2019, Sarica et al., 2023, Han et al., 2020).

1. Data Sources and Network Construction

The construction of TechNet begins with the ingestion of the entire corpus of U.S. utility patents, spanning 1976–2019 (>5.7 million patents, ≈26.8 million sentences, ≈699 million word tokens). Source data from patent titles, abstracts, claims, and descriptions are preprocessed by NLP pipelines involving sentence and word tokenization, POS tagging, lemmatization, and an expanded list of stop words tailored to the patent domain. Candidate technical terms—unigrams, bigrams, trigrams, and four-grams—are extracted via statistical phrasing (Mikolov et al. 2013) and filtered using document-frequency criteria (DF ≥ 50) to ensure statistical reliability. The final vocabulary size reaches ≈4 million unique, domain-significant terms (Sarica et al., 2019, Sarica et al., 2020).

Terms are then embedded as 300-dimensional vectors using the SGNS (Skip-gram with Negative Sampling) Word2Vec model. The context window (typically c = 20) and downsampling parameters are optimized for broad coverage and semantic discrimination of technical terms. Training is performed on the entire preprocessed corpus, with each technical term token treated uniformly (no distinction between components, functions, or materials). The result is an embedding space in which semantic relatedness can be efficiently computed (Sarica et al., 2020).

Edges between nodes are defined by cosine similarity: $$\mathrm{sim}(i,j) = \cos(\mathbf{v}_i, \mathbf{v}_j) = \frac{\mathbf{v}_i^\top \mathbf{v}_j}{\|\mathbf{v}_i\|\, \|\mathbf{v}_j\|}$$ Only pairs exceeding a chosen similarity threshold (typically θ ≈ 0.6) are retained as edges, resulting in ≈62 million edges in the initial explicit subnetwork, with most weights concentrated in [0.6, 0.8] and a small number indicating near-synonymy (weights > 0.9) (Han et al., 2020).

2. Schema, Statistics, and Structural Properties

TechNet employs a flat schema: all nodes are technology concepts, and all edges are weighted undirected semantic similarity relations. No handcrafted hierarchy (e.g., “is-a”, “part-of”) is imposed; all structure arises directly from statistical co-occurrence (Han et al., 2020).

Key statistics:

• Nodes: |V| ≈ 4,038,924
• Edges: |E| ≈ 62,000,000 (explicit above similarity threshold), sparsely materialized
• Potential term–term pairs: |V| · (|V|−1)/2 ≈ 8.2×10¹²
• Average degree (thresholded network): ≈ 30, with a heavy-tailed distribution indicating the presence of domain hubs (e.g., “system”, “method”, “control”)
• Clustering coefficient is large at thresholded scales, producing local small-world properties (Sarica et al., 2019, Han et al., 2020).

The network aligns exhaustively with the patent lexicon, capturing complex, multi-word technical terms neglected by general language resources.

3. Algorithms, Retrieval, and Graph Construction Methods

TechNet supports diverse retrieval and knowledge representation tasks. For document-driven design mapping, the following procedure is used (Sarica et al., 2020):

1. Extract terms present in a given document D to form a candidate node set V.
2. Construct an adjacency matrix A where A_{ij} = sim(t_i, t_j) for each pair of extracted terms.
3. Build a maximum spanning tree (using Kruskal’s algorithm) to enforce overall graph connectivity with |V|–1 edges.
4. Augment the MST by incrementally adding the highest-weight residual edges until reaching 2|V| total edges, balancing graph density and readability.
5. Apply a force-directed layout (e.g. Fruchterman–Reingold, ForceAtlas2) for visual summary.

Term similarity and retrieval rely on cosine similarity of learned embeddings. For baseline comparisons, WordNet path similarity and ConceptNet cosine similarity are used, but coverage and accuracy in technical domains are substantially lower. TechNet also exposes a REST API (http://www.Tech-Net.org) supporting on-demand similarity lookups, neighborhood expansion, and interactive graph visualizations (Sarica et al., 2019).

4. Evaluation, Comparative Performance, and Human Studies

TechNet’s effectiveness has been assessed both intrinsically (term coverage and retrieval quality) and extrinsically (impact on engineering tasks):

• On general engineering term recall, TechNet reports ≈0.72 versus 0.40 for WordNet and 0.64 for ConceptNet.
• On a custom “Technical Term Relevance” (TTR) dataset, TechNet achieves Spearman’s ρ ≈ 0.66, outperforming both general-language and prior domain embeddings (Sarica et al., 2019).
• In human-subject studies with engineering experts, TechNet-based design graphs are rated as more representative of technical meanings and subsystem structure. For instance, in the “spherical robot” case, ~70% of subjects selected TechNet maps as best, compared to ~20% for WordNet and ~10% for ConceptNet (Sarica et al., 2020).
• On document or query expansion tasks, TechNet yields up to 12% higher recall@20 than WordNet-augmented baselines, and 8% higher precision@20 than ConceptNet (Han et al., 2020).
• Downstream application studies demonstrate improved MAP (mean average precision) in patent search (+0.07 when adding TechNet expansions) and improved F1 in technology cluster forecasting (+15%) (Han et al., 2020).

Table: Comparative Coverage and Human Study Outcomes

Source	Term Coverage (Recall)	Rated Best in Human Study
TechNet	≈ 0.72	~70%
WordNet	≈ 0.40	~20%
ConceptNet	≈ 0.64	~10%

5. Applications in Engineering Knowledge Discovery and Innovation Analysis

TechNet serves as core infrastructure for a portfolio of applications across engineering research and design:

• Technical text summarization and mapping: Enables abstraction of lengthy patent or design documents into entity–relation graphs preserving subsystem boundaries and terminology precision.
• Search/query expansion: Supplies semantically related terms and synonyms for more effective prior art and literature searches.
• Relational knowledge discovery: Supports analogy mining and non-obvious technology mapping via multi-hop graph exploration.
• Design ideation: Facilitates combinatorial exploration of concept neighborhoods, supporting divergent and analogical generation of ideas.
• Patent landscaping: Visualizes technology fields, gaps, and innovation trajectories for R&D strategy (Sarica et al., 2019, Sarica et al., 2020, Han et al., 2020).

Crucially, TechNet enables the empirical study of the “technology concept space” and the dynamics of innovation itself. Analysis reveals:

• The cumulative number of patented concepts S(t) grows linearly over time—not exponentially.
• The originality of new concepts, as measured by their information-theoretic distance from prior art, is steadily declining; new terms are increasingly similar to existing ones: $$AIC(x_{\mathrm{new}}) \approx \log\frac{1}{\cos(\mathbf{x}_{\mathrm{new}}, \mathbf{p}^*)}$$ where $\mathbf{p}^*$ is the closest prior concept (Sarica et al., 2023). These findings indicate strong negative feedback in the innovation process, likely due to cognitive and information overload constraints (Sarica et al., 2023).

6. Limitations, Enhancement Opportunities, and Future Directions

Limitations of the current TechNet release include the absence of explicit ontological (taxonomic or partonomic) relations and constrained handling of multimodal patent data (such as figures or diagrams). The static Word2Vec embedding may miss subtle contextual distinctions or shifts over time.

Prioritized enhancement directions include:

• Integration of patent images, structural diagrams, and claim graphs for richer multi-modal node annotation.
• Incremental and real-time updates as new patents issue, tackling temporal drifts and emergent concept discovery.
• Augmentation with transformer-based, contextual LLMs (e.g., patent-specialized BERT or GPT) to capture finer semantic nuance.
• Overlaying expert-verified relational data (“is-a”, “has-part”, etc.) to improve interpretability and semantic search (Han et al., 2020).

A plausible implication is that hybridizing TechNet’s data-driven graph with curated ontological layers may yield knowledge representations optimal for both retrieval accuracy and explainability.

Creative Artificial Intelligence (CAI) methods built on TechNet are being investigated to counteract the decline in conceptual originality by generating and evaluating highly novel, technically plausible concepts. Empirical evidence suggests fine-tuned generative models can propose patentable ideas that extend or recombine distant regions in the TechNet graph, presenting opportunities for accelerating innovation beyond human cognitive limits (Sarica et al., 2023).

7. Resources and Access

TechNet is accessible through a RESTful API and browser-based web portal at http://www.Tech-Net.org, offering endpoints for term similarity lookup, neighborhood expansion, technical text mapping, and visualization (outputs in JSON/CSV and interactive D3.js). The underlying database supports programmatic research utilization at scale (Sarica et al., 2019). The code and pretrained models are publicly available, ensuring reproducibility and extensibility in scholarly applications.

In summary, TechNet constitutes the first comprehensive, patent-derived semantic network tailored for engineering and technology, characterized by large-scale data ingestion, statistically grounded construction, and high domain suitability. It provides the foundation for advanced retrieval, mapping, and ideation capabilities in engineering research and empirical analyses of the evolution of the technological concept space (Sarica et al., 2020, Sarica et al., 2019, Sarica et al., 2023, Han et al., 2020).