---
title: 'GRASP-1k: Robotic Grasping & Geospatial Benchmark'
url: https://www.emergentmind.com/topics/grasp-1k
type: topic
---

# GRASP-1k: Robotic Grasping & Geospatial Benchmark

GRASP-1k denotes two distinct research datasets introduced in different arXiv works: a robotic grasping dataset associated with the Grasp Reset Mechanism (GRM) in "The Grasp Reset Mechanism: An Automated Apparatus for Conducting Grasping Trials" [2402.18650], and a geospatial reasoning-and-segmentation benchmark introduced in "GRASP: Geospatial pixel Reasoning viA Structured Policy learning" [2508.17102]. In the first usage, GRASP-1k is a corpus of 1,020 physical grasp trials collected with a fully automated robotic apparatus; in the second, it is an out-of-domain benchmark of 1,071 reasoning-segmentation examples for natural-language-guided geospatial mask generation. The shared name reflects different senses of “grasp”: physical manipulation in robotics and structured grounding in spatially localized vision-language reasoning.

## 1. Nomenclature and scope

The term GRASP-1k is not unique to a single benchmark. In the available literature, it names two separate datasets with different modalities, objectives, and evaluation regimes.

| Usage | Paper | Core content |
|---|---|---|
| Robotic grasping | [2402.18650] | 1,020 physical grasp trials collected with the GRM |
| Geospatial pixel reasoning | [2508.17102] | 1,071 reasoning-segmentation examples for OOD evaluation |

In the robotic setting, the dataset is explicitly tied to a hardware-software platform for automated resetting, object swapping, and repeatable grasp execution [2402.18650]. In the geospatial setting, the benchmark supports reinforcement learning for segmentation from language instructions, with structured outputs consisting of bounding boxes and points rather than direct mask supervision [2508.17102].

This suggests that GRASP-1k is best understood as a context-dependent dataset name whose meaning must be resolved from the associated paper and task domain.

## 2. GRASP-1k in robotic grasping research

In [2402.18650], GRASP-1k is a dataset of 1,020 grasps created with a Kinova Gen3 robot arm and Robotiq 2F-85 Adaptive Gripper. The stated purpose is to enable training of learning models and to demonstrate the capabilities of the GRM. The dataset includes ranges of grasps conducted across four objects and a variety of orientations, with manipulator states, object pose, video, and grasp success data provided for every trial.

The overall statistics reported are 1,020 total trials and 715 successful grasps, approximately 70% [2402.18650]. The objects and orientations are specified as follows: a rectangular prism of \(40\times 40\times 105\) mm with \(\theta\in\{0^\circ,15^\circ,30^\circ,45^\circ\}\); a triangular prism of \(50\times 50\times 105\) mm with \(\theta\in\{0^\circ,20^\circ,40^\circ,60^\circ\}\); a cylinder of \(\varnothing 40\times 105\) mm with \(\theta=\{0^\circ\text{ only}\}\); and a cone with base \(\varnothing 40\), height \(105\) mm, again with \(\theta=\{0^\circ\text{ only}\}\). All shapes are rigid, with a magnetic base insert and an on-top ArUco marker, except that the cone uses a colored dot [2402.18650].

For each \((\text{object},\theta)\) combination, 15 equally spaced deviations were applied in one degree of freedom of the gripper. The perturbations include translations \(\Delta x\in[0,3\text{ cm}]\) and \(\Delta y\in[1,5\text{ cm}]\), as well as rotations in roll about \(X\) over \([0^\circ,45^\circ]\), pitch about \(Y\) over \([0^\circ,90^\circ]\), and yaw about \(Z\) over \([0^\circ,60^\circ]\). Two grasp types are represented: top-down and side-approach [2402.18650].

The recorded modalities include manipulator joint states, comprising positions and velocities at 100 Hz; gripper finger widths and forces; object pose from an ArUco tracker or color-dot tracker at 30 Hz; top-view RGB at 30 Hz; side-view RGB at 30 Hz; a wrist-mounted Intel RealSense RGB-D stream at 30 Hz; and a binary success label per trial [2402.18650].

The paper also reports sample statistical highlights. For top grasps under X-translation, success rates are 68% for the rectangle, 35% for the triangle, 53% for the cylinder, and 40% for the cone. For top grasps under Y-rotation, the rates are 95%, 90%, 47%, and 33%, respectively. For side grasps under X-rotation, the corresponding rates are 100%, 97%, 93%, and 100% [2402.18650].

## 3. GRM apparatus, reset pipeline, and software interface

The robotic GRASP-1k dataset is inseparable from the Grasp Reset Mechanism. The GRM is described as a fully automated apparatus for conducting large-scale grasping trials, automating the process of resetting a grasping environment, repeatably placing an object in a fixed location and controllable 1-D orientation, collecting data, and swapping between multiple objects with no human intervention [2402.18650].

Its lower reset subsystem consists of a centering cone driven by a vertical ballscrew and NEMA-17 stepper, a retractable string driven by a second NEMA-17 stepper with a gravity-loaded tensioner, and a rotating platform of 25 cm diameter driven by a DC motor with a quadrature-style encoder and a Hall-effect “zero” sensor [2402.18650]. Electrical home-position sensing is implemented with a ballscrew limit switch for cone-up detection and copper-plate contacts around the cone that detect when the magnetic insert on the object draws current, indicating that the object is seated and centered [2402.18650].

The upper reset, or object swapping subsystem, is a back-mounted 3-DOF mini-gantry with two linear NEMA-17 steppers for \(X\) and \(Z\), one rotary stepper about \(Z\), and an electromagnet at its wrist. The described procedure is: the lower reset lifts the object on its cone, the upper arm swings in and powers the electromagnet, lifts the object off the string insert, places it in a storage bin, picks up the next pre-loaded object by magnet, and returns it to the cone [2402.18650].

The control architecture is a distributed ROS-based system tested under Melodic and Noetic, with two logical hosts. A master control computer runs a FlexBE hierarchical state machine, arm action servers and clients, and data-collection nodes, while an on-GRM Raspberry Pi with two Arduinos handles stepper drivers, an H-bridge, limit-switch interrupts, and electromagnet control [2402.18650].

The high-level behavior is decomposed into FlexBE states: TestControl, TrialControl, ResetEnvironment, ExecuteGrasp, DataCollection, EvaluateSuccess, and LogResults [2402.18650]. Example ROS interfaces include `/grm/lower_reset` with goal `{angle: θ}`, `/grm/swap_object` with goal `{next_object_id}`, `/arm/execute_grasp` with goal `{SE(3) pose}`, and `/data_recorder/start_stop` for rosbag control [2402.18650]. The open-source materials include CAD and a bill of materials, and the software interface exposes ROS action servers such as `grm_lower_reset.action`, `grm_rotate.action`, `grm_swap_object.action`, `arm_execute_grasp.action`, and `arm_set_home.action` [2402.18650].

The reset precision is quantified in two ways. The centering cone gives an accurate 2-D XY home location with \(<0.1\) mm \(\sigma\), and the platform rotates to target \(\theta\) about its vertical axis with \(\sigma_\theta\approx 1.3^\circ\) [2402.18650]. The paper also reports pose repeatability from 20 resets tracked with ArUco as \(\sigma_{xy}\approx 0.02\) mm and \(\sigma_\theta\approx 1.3^\circ\) [2402.18650].

## 4. Formal task definition and evaluation in the robotic dataset

The robotic paper specifies a grasp representation
$$
g=(R,t)\in SE(3),
$$
where
$$
R=R_z(\theta)\cdot R_y(\phi)\cdot R_x(\psi)
$$
and
$$
t=(x,y,z)^\top.
$$
Within the dataset, only \(\theta\) around world-\(Z\) is varied via the GRM, while \(\phi\) and \(\psi\) remain at zero. End-effector perturbations add \(\delta\in\mathbb{R}^1\) along one axis or one Euler angle [2402.18650].

The success criterion is defined during a lift-and-move to a fixed target 25 cm away. A grasp is labeled successful, \(s=1\), if and only if two conditions both hold: the gripper never fully closes, meaning the width never equals zero, and the final object centroid at the target lies within \(\epsilon=5\) mm. Formally,
$$
s = 1\!\left[\left(\min_t w_{\text{finger}}(t)>0\right)\wedge\left(\|p_{\text{obj,end}}-p_{\text{target}}\|<\epsilon\right)\right].
$$
The evaluation metrics include success rate per \((\text{object},\theta,\text{perturbation},\text{grasp\_type})\), transition boundary \(\theta^\*\) or \(\delta^\*\) where success transitions to failure, and pose repeatability [2402.18650].

The usage notes identify several experimental roles for GRASP-1k. These include supervised learning, such as training a CNN to predict \(s\) from RGB/D and pose; boundary estimation, namely learning \(\delta^\*\) as a function of object shape and \(\theta\); domain adaptation, specifically sim2real bootstrapping using fine angle increments; and benchmarking new grasp planners on the same 1,020-trial protocol [2402.18650]. A plausible implication is that the dataset was designed not only for aggregate success-rate reporting, but also for sensitivity analysis around failure boundaries and controlled physical repeatability.

## 5. GRASP-1k in geospatial pixel reasoning

In [2508.17102], GRASP-1k is a benchmark for geospatial pixel reasoning, a task defined as generating segmentation masks directly from natural-language instructions. The paper describes prevailing multimodal large-language-model systems as co-training a language model and a mask decoder with dense pixel supervision, then introduces GRASP as a structured policy-learning framework in which a multimodal large language model emits task-relevant bounding boxes and positive points from a vision-language instruction, and a pre-trained segmentation model consumes them as prompts to generate the final mask [2508.17102].

The benchmark itself contains 1,071 reasoning-segmentation examples. Its source imagery comes from six out-of-domain aerial or spaceborne collections: CVUSA, NWPU-RESISC45, CrowdAI, fMoW, CVACT, and LoveDA [2508.17102]. The initial pool comprises 2,000 images sampled per source, for 12,000 total, after which a quality filter retains images satisfying \(\mathrm{BRISQUE}(I)<50\), written as
$$
\mathrm{BRISQUE}(I)=f(\phi(I)), \qquad
I_{\text{clean}}=\{\,I\mid \mathrm{BRISQUE}(I)<50\,\}.
$$
After automated prompting and human curation, the final size is 1,071 [2508.17102].

Each sample contains one natural-language question requiring multi-step geospatial reasoning. The examples given include directional queries such as “Which road lies immediately north of the river bend?”, relational or comparative queries such as “Identify the largest cluster of shipping containers.”, and contextual queries such as “Find the building that is shaded by the tallest tree.” All prompts are generated by Gemini-2.5-Pro and vetted by human annotators to ensure logical complexity [2508.17102].

Every example includes a chain-of-thought wrapped in explicit tags: `<think> ... </think>` encloses stepwise natural-language reasoning, and `<answer> ... </answer>` encloses the final grounding in spatial terms. Inside `<answer>`, the required output schema is a bounding box
$$
B=[x_{\min},y_{\min},x_{\max},y_{\max}],
$$
together with two points, \(\mathrm{Point\ 1}=(x_1,y_1)\) and \(\mathrm{Point\ 2}=(x_2,y_2)\) [2508.17102]. The paper states that this strict schema enables automatic parsing during RL training.

The segmentation annotations are fine-grained masks \(M\), binary or polygonal, produced by SAM2 or manually in LabelMe. The ground-truth bounding box \(B_g\) is extracted from \(M\) as
$$
B_g=(x_1^g,y_1^g,x_2^g,y_2^g),
$$
with \(w_g=x_2^g-x_1^g\) and \(h_g=y_2^g-y_1^g\). Two positive points are defined: \(p_1^g\), the center of the maximal inscribed circle in \(M\), and \(p_2^g\), a supplementary point on the mask boundary, described as an outer-ring or farthest-boundary pixel [2508.17102].

The split definition is explicit. Training and in-domain test sets are EarthReason, using Million-AID and DIOR, and GeoPixInstruct, using HRSC2016, DOTA-V2.0, and FAIR1M-2.0. The OOD test set is GRASP-1k itself, and no part of GRASP-1k is used during RL training; it serves purely for robustness evaluation [2508.17102].

## 6. Reinforcement-learning signals and benchmark role in the geospatial setting

The core idea of GRASP is to learn bounding-box-plus-point grounding with reinforcement learning from weak spatial cues instead of full masks. For each training question \(Q\) and image \(I\), the multimodal large language model outputs \((B_p,p_1^p,p_2^p)\), and rewards are computed against \((B_g,p_1^g,p_2^g)\) [2508.17102].

Format rewards, denoted \(R_{\mathrm{fmt}}\), assign 1 point if a well-formed `<think>...</think>` chain is present and 1 point if the `<answer>` block exactly follows the bounding-box/point schema [2508.17102]. Accuracy rewards, denoted \(R_{\mathrm{acc}}\), measure grounding quality via bounding-box IoU, a soft scale-normalized box-distance term, and point accuracy. The paper specifies
$$
\mathrm{IoU}(B_p,B_g)=\frac{\mathrm{area}(B_p\cap B_g)}{\mathrm{area}(B_p\cup B_g)},
$$
with reward
$$
R_{\mathrm{iou}}=1 \ \text{if}\ \mathrm{IoU}(B_p,B_g)>0.5\ \text{else}\ 0.
$$
The normalized center distance is
$$
d_{\mathrm{bbox}}=\frac{1}{2}\left(\frac{|c_x^p-c_x^g|}{w_g}+\frac{|c_y^p-c_y^g|}{h_g}\right),
$$
and the corresponding reward is
$$
R_{\mathrm{bbox-dist}}=\max\left(0,1-\frac{d_{\mathrm{bbox}}}{0.5}\right).
$$
For the points,
$$
d_1=\frac{\lVert p_1^p-p_1^g\rVert_1}{s_{\min}}, \qquad
d_2=\frac{\lVert p_2^p-p_2^g\rVert_1}{s_{\max}}, \qquad
S=\frac{d_1+d_2}{2},
$$
with
$$
R_{\mathrm{pts}}=1 \ \text{if both}\ p_i^p\ \text{lie inside}\ B_g\ \text{and}\ S<0.5,\ \text{else}\ 0.
$$
These rewards drive Grouped Relative Policy Optimization (GRPO) [2508.17102].

The paper reports that evaluations on both in-domain and out-of-domain test sets show state-of-the-art results, with about 4% improvement in-domain and up to 54% on OOD benchmarks [2508.17102]. Within the scope of GRASP-1k specifically, the benchmark is positioned as a robustness test set rather than a training corpus, and the detailed reasoning traces, structured grounding outputs, and fine-grained masks make it suitable for evaluating OOD generalization in geospatial pixel reasoning [2508.17102].

## 7. Comparative interpretation and research significance

The two GRASP-1k datasets occupy very different methodological niches. The robotic GRASP-1k is a physical-interaction dataset built around repeatable object placement, controllable 1-D orientation, and tightly instrumented trial logging [2402.18650]. The geospatial GRASP-1k is a reasoning-intensive OOD benchmark organized around language-conditioned grounding, structured intermediate outputs, and reinforcement-learning-compatible reward definitions [2508.17102].

Despite the domain difference, both resources formalize an intermediate representation that constrains the task. In robotics, the grasp is parameterized as \(g=(R,t)\in SE(3)\) with controlled variation primarily in \(\theta\) and perturbation \(\delta\) [2402.18650]. In geospatial reasoning, the model must emit a bounded schema consisting of a box and two points before mask generation [2508.17102]. This suggests a shared design pattern: rather than treating the final task outcome as monolithic, both benchmarks expose structured variables that support fine-grained evaluation.

They also differ in what “benchmarking” means. For the robotic dataset, benchmarking refers to comparing grasp planners on the same 1,020-trial protocol and quantifying success boundaries, repeatability, and success rates under controlled perturbations [2402.18650]. For the geospatial dataset, benchmarking refers to robustness evaluation on an OOD set that is excluded from RL training, with performance judged through spatial grounding quality and downstream segmentation behavior [2508.17102].

A common misconception would be to treat GRASP-1k as a single canonical benchmark. The literature does not support that reading. Instead, GRASP-1k refers to at least two separate datasets: one for automated robotic grasp trials and one for geospatial pixel reasoning. Accurate citation therefore requires pairing the name with its paper identifier, either [2402.18650] or [2508.17102], and with the surrounding task context.

Source: https://www.emergentmind.com/topics/grasp-1k