Utilizing Synthetic Data in Supervised Learning for Robust 5-DoF Magnetic Marker Localization (2211.07556v2)
Abstract: Tracking passive magnetic markers plays a vital role in advancing healthcare and robotics, offering the potential to significantly improve the precision and efficiency of systems. This technology is key to developing smarter, more responsive tools and devices, such as enhanced surgical instruments, precise diagnostic tools, and robots with improved environmental interaction capabilities. However, traditionally, the tracking of magnetic markers is computationally expensive due to the requirement for iterative optimization procedures. Moreover, these methods depend on the magnetic dipole model for their optimization function, which can yield imprecise outcomes due to the model's significant inaccuracies when dealing with short distances between non-spherical magnet and sensor.Our paper introduces a novel approach that leverages neural networks to bypass these limitations, directly inferring the marker's position and orientation to accurately determine the magnet's 5 DoF in a single step without initial estimation. Although our method demands an extensive supervised training phase, we mitigate this by introducing a computationally more efficient method to generate synthetic, yet realistic data using Finite Element Methods simulations. The benefits of fast and accurate inference significantly outweigh the offline training preparation. In our evaluation, we use different cylindrical magnets, tracked with a square array of 16 sensors. We perform the sensors' reading and position inference on a portable, neural networks-oriented single-board computer, ensuring a compact setup. We benchmark our prototype against vision-based ground truth data, achieving a mean positional error of 4 mm and an orientation error of 8 degrees within a 0.2x0.2x0.15 m working volume. These results showcase our prototype's ability to balance accuracy and compactness effectively in tracking 5 DoF.
- Hedgehog: Handheld spherical pin array based on a central electromagnetic actuator. In 2021 IEEE World Haptics Conference (WHC), pages 133–138. IEEE, 2021.
- Automatic differentiation in machine learning: a survey. Journal of machine learning research, 18, 2018.
- KW Brodlie and Sohail Butt. Preserving convexity using piecewise cubic interpolation. Computers & Graphics, 15(1):15–23, 1991.
- Magx: Wearable, untethered hands tracking with passive magnets. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, pages 269–282, 2021.
- utrack: 3d input using two magnetic sensors. In Proceedings of the 26th annual ACM symposium on User interface software and technology, pages 237–244, 2013.
- Finexus: Tracking precise motions of multiple fingertips using magnetic sensing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 1504–1514, 2016.
- A deep learning framework for solution and discovery in solid mechanics. arXiv preprint arXiv:2003.02751, 2020.
- Wearable handwriting input device using magnetic field. In SICE Annual Conference 2007, pages 365–368. IEEE, 2007.
- Wireless magnetic motion capture system for multi-marker detection. IEEE transactions on magnetics, 42(10):3279–3281, 2006.
- Efficient magnetic localization and orientation technique for capsule endoscopy. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 628–633, 2005.
- Adam: A method for stochastic optimization, 2017.
- Modality-integrated magnetic catheter tracking for x-ray vascular interventions. Physics in Medicine & Biology, 50(4):581, 2005.
- Gaussmarbles: spherical magnetic tangibles for interacting with portable physical constraints. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 4228–4232, 2016.
- Omni: Volumetric sensing and actuation of passive magnetic tools for dynamic haptic feedback. 2020.
- Optimal control for electromagnetic haptic guidance systems. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, pages 951–965, 2020.
- Magnetic object positioning based on second-order magnetic gradient tensor system. IEEE Sensors Journal, 21(16):18237–18248, 2021.
- Gausssense: attachable stylus sensing using magnetic sensor grid. In Proceedings of the 25th annual ACM symposium on User interface software and technology, pages 319–326, 2012.
- On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1):503–528, 1989.
- Evolutionarily optimized electromagnetic sensor measurements for robust surgical navigation. IEEE Sensors Journal, 19(22):10859–10868, 2019.
- Magnetic hand tracking for human-computer interface. IEEE Transactions on Magnetics, 47(5):970–973, 2011.
- Plasmonic nanostructure design and characterization via deep learning. Light: Science & Applications, 7(1):1–8, 2018.
- Magnetips: Combining fingertip tracking and haptic feedback for around-device interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–12, 2019.
- Simulation of hyperelastic materials in real-time using deep learning. Medical image analysis, 59:101569, 2020.
- Auraring: Precise electromagnetic finger tracking. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(4):1–28, 2019.
- Optimal permanent-magnet geometries for dipole field approximation. IEEE Transactions on Magnetics, 49(2):811–819, 2013.
- Grzegorz Psuj. Multi-sensor data integration using deep learning for characterization of defects in steel elements. Sensors, 18(1):292, 2018.
- Timo Pylvänäinen. Automatic and adaptive calibration of 3d field sensors. Applied Mathematical Modelling, 32(4):575–587, 2008.
- Force calculation for localized magnetic driven capsule endoscopes. In 2013 IEEE International Conference on Robotics and Automation, pages 5354–5359. IEEE, 2013.
- Magnetic-field-based position sensing using machine learning. IEEE Sensors Journal, 20(13):7292–7302, 2020.
- A deep neural network-based permanent magnet localization for tongue tracking. IEEE Sensors Journal, 19(20):9324–9331, 2019.
- A novel passive magnetic localization wearable system for wireless capsule endoscopy. IEEE Sensors Journal, 19(9):3462–3472, 2019.
- A 5-d localization method for a magnetically manipulated untethered robot using a 2-d array of hall-effect sensors. IEEE/ASME Transactions on Mechatronics, 21(2):708–716, 2015.
- Real-time multi-object magnetic tracking for multi-arm continuum robots. IEEE Transactions on Instrumentation and Measurement, 70:1–9, 2021.
- 6-d magnetic localization and orientation method for an annular magnet based on a closed-form analytical model. IEEE Transactions on Magnetics, 50(9):1–11, 2014.
- Amagposenet: Real-time 6-dof magnet pose estimation by dual-domain few-shot learning from prior model. IEEE Transactions on Industrial Informatics, 2023.
- Passive magnetic-based localization for precise untethered medical instrument tracking. Computer methods and programs in biomedicine, 156:151–161, 2018.
- Enhanced real-time pose estimation for closed-loop robotic manipulation of magnetically actuated capsule endoscopes. The International journal of robotics research, 37(8):890–911, 2018.
- Low-latency tracking of multiple permanent magnets. IEEE Sensors Journal, 19(23):11458–11468, 2019.
- Magnetomicrometry. Science robotics, 6(57):eabg0656, 2021.
- A review of localization systems for robotic endoscopic capsules. IEEE Transactions on Biomedical Engineering, 59(9):2387–2399, 2012.
- A fast and robust magnetic localization technique based on elimination of the orientation variables from the optimization. IEEE Sensors Journal, 21(19):21885–21892, 2021.