Skip to content
jioyPublic

About

Toward Human-Level Tactile Perception: A Large-Scale, High-Density Iontronic Skin This project is developing a large-scale, high-density, and fast-response iontronic skin for human-level tactile perception in robotics.

Resources

Stars

24 stars

Watchers

0 watching

Forks

Repository files navigation

Large-Scale Iontronic Skins with Neural-Hierarchical Processing Architecture

Python PyTorch License Platform





📑 Table of Contents


📖 Overview

NI-skin is a large-scale neural-hierarchical iontronic skin system that mimics the hierarchical processing architecture of the human tactile sensory system. The human skin encodes and gathers signals from over 200,000 afferents with high spatiotemporal resolution — a capability that existing serial-architecture robotic skins cannot match when scaled to thousands of receptors due to readout latency, signal crosstalk, and bandwidth consumption.

Our system introduces a three-layer architecture — receptor, spinal, and cognitive layers — that processes tactile information through frequency-based position encoding, parallel transmission, backend convergent compression, and cognitive modeling:

  • Receptor Layer — Ionotronic sensing elements position-encoded with optimized driving frequencies for parallel, crosstalk-free signal transmission
  • Spinal Layer — Signal decoding, convergence, and 3D-DWT compression to minimize bandwidth
  • Cognitive Layer — 3D-CNN-based spatiotemporal feature extraction for tactile recognition and pain perception

The system integrates >10,000 sensing elements with a minimum inter-sensor pitch of 300 μm (1,110 units cm⁻²) and a temporal resolution of 10 ms, enabling robots to perform fine tactile interactions and proactive injury avoidance through nociceptive sensing.

Ultra-High-Density Tactile Perception

With a sensor density of 1,110 units cm⁻² — exceeding the mechanoreceptor density of the human fingertip (~240 units cm⁻²) — the NI-skin resolves fine surface textures of fabrics and coins at a feature size down to 300 μm.

Dexterous Fine Manipulation

The high-sensitivity, high-density NI-skin enables robots to delicately handle slender and fragile objects such as pencil leads (0.5 mm diameter), soft solder wires (0.2 mm diameter), and spiny chestnuts — tasks that demand fine tactile feedback and real-time pressure monitoring.

Nociceptive Sensing & Self-Protection

The NI-skin quantifies sharp-pain stimuli using a Logit regression model (R² = 91.1%) that fuses maximum pressure, activated-element count, and maximum pressure gradient. When the sensed pressure gradient exceeds the pain threshold, the robot triggers a protective withdrawal or grasp-release reflex to prevent skin damage — mimicking the human nociceptive response.


✨ Key Features

Feature Description
Neural-Hierarchical Architecture Three-layer design (receptor → spinal → cognitive) emulating the human tactile nervous system for efficient parallel processing
🔬 Ultra-High Sensor Density Minimum 300 μm inter-sensor pitch (1,110 units cm⁻²), exceeding the mechanoreceptor density of the human fingertip (~240 units cm⁻²)
Parallel Frequency Encoding 96 receptors per neural bundle, each encoded at 1–20 kHz with 200 Hz intervals; orthogonal frequency-division enables crosstalk-free parallel readout
PZP Crosstalk Suppression Parallel zero-potential (PZP) mechanism reduces crosstalk from ~39.4% to ~1.3%, simultaneously suppressing both ionic and electrical interference
3D-DWT Compressed Sensing Three-dimensional discrete wavelet transform compression achieves CR = 268 (SSIM > 0.97), reducing bandwidth from 14.7 Mbps to 56.7 kbps
Pain Perception & Self-Protection Logit regression-based pain model (R² = 91.1%) detects sharp stimuli and triggers protective reflexes for proactive injury avoidance
Scalable Multi-Scale Integration NI-skins in multiple form factors (24×24 to 96×96 elements) deployed on fingertip, hand, and whole-body robotic surfaces
Real-Time Pipeline End-to-end latency of ~23 ms (sampling → FFT → compression → transmission → inference), supporting closed-loop robotic control

System Architecture

The NI-skin system emulates the hierarchical tactile processing of the human body across three functional layers:

┌─────────────────────────────────────────────────────────┐
│  COGNITIVE LAYER (Central Processor / GPU)              │
│  3D-CNN inference · Tactile recognition · Pain model    │
│  Lightweight network: 938M FLOPS, 3.6K parameters       │
├─────────────────────────────────────────────────────────┤
│  SPINAL LAYER (FPGA Edge Processor)                     │
│  FFT demodulation · PZP crosstalk suppression           │
│  3D-DWT compression (CR = 268) · Data aggregation       │
├─────────────────────────────────────────────────────────┤
│  RECEPTOR LAYER (NI-skin Sensor Array)                  │
│  >10,000 iontronic sensors · Frequency-encoded (1–20 kHz)│
│  300 μm pitch · 100 Hz readout · <600 μm thickness      │
└─────────────────────────────────────────────────────────┘

Key Performance Metrics

Metric Value
Sensor density 1,110 units cm⁻²
Total sensing elements >10,000
Minimum inter-sensor pitch 300 μm
Temporal resolution (readout) 10 ms (100 Hz)
Frequency-bin resolution 100 Hz (512-pt FFT)
Adjacent leakage ratio (ALR) <1%
Crosstalk (with PZP) ~1.3%
Bending radius 1.2 mm
Skin thickness <600 μm
Operating pressure range 0–100 kPa (linear), up to 550 kPa
Detection limit <0.08 g
Two-point limen 0.8 mm
Cycling stability >100,000 cycles
Drift (100 kPa load) ~9.8%
Compression ratio (optimal) 268× (SSIM > 0.97)
Bandwidth reduction 14.7 Mbps → 56.7 kbps
Grasp recognition accuracy >98.2% (10 classes)
Gesture recognition accuracy >95.4% (10 classes)
Pain model R² 91.1%
End-to-end pipeline latency ~23 ms

Repository Structure

NI-skins/
├── Hardware Design/                    # Firmware, PCB schematics & circuit documentation
│   └── Circuit schematic diagram/
│       └── Schematic diagram.pdf
│
├── Graphical User Interface (GUI)/     # Python-based sensor visualization & data acquisition
│   ├── 1.Sensor array_3DMAP_Demo/      # 3D tactile map visualization
│   └── 2.Sensor_array_2Dmapping/       # 2D tactile mapping interface
│
├── Deep Learning Framework/            # PyTorch-based neural network models
│   ├── 1.TactileCompress_evaluation/   # 3D-DWT compression evaluation (CR vs. SSIM)
│   ├── 2.Object recognition/           # 3D-CNN for object classification (10 classes)
│   ├── 3.Gesture recognition/          # 3D-CNN for gesture classification (10 classes)
│   └── 4.Pain_model/                  # Logit regression for pain estimation (5 levels)
│
├── Dexterous Hand Control Scripts/     # Real-time sensor→robot closed-loop control
│   └── 1.Soft robotic finger grasping/ # Gripper control via USB/TCP tactile feedback
│
├── assets/                             # Images, GIFs, and figures
├── LICENSE                             # MIT License
└── README.md

Quick Start

Prerequisites

  • OS: Windows
  • Python: 3.9+
  • IDEs: STM32CubeMX / STM32CubeIDE / Keil5 / Quartus II / VS Code

Python Dependencies

pip install torch torchvision numpy matplotlib seaborn scikit-learn \
            h5py pandas statsmodels tensorboardX tqdm PyWavelets thop

Setup

  1. Clone the repository

    git clone https://github.com/jioy/NI-skins.git
    cd NI-skins
  2. Download datasets — unzip all data folders and place them under the corresponding Alldata/ directory.

  3. Update data paths — scripts contain default paths; update them to match your local machine before running.

Usage Examples

Compression evaluation (sweep Q factors for CR–SSIM trade-off):

cd "Deep Learning Framework/1.TactileCompress_evaluation"
python Compression_evaluation.py

Object recognition training (3D-CNN, 10-class grasping):

cd "Deep Learning Framework/2.Object recognition"
python train.py --compress_q 2000 --train_log_dir ./result

Gesture recognition training (3D-CNN, 10-class touch gestures):

cd "Deep Learning Framework/3.Gesture recognition"
python train.py --compress_q 2001 --train_log_dir ./result

Pain estimation (Logit regression, 5 sharpness levels):

cd "Deep Learning Framework/4.Pain_model"
python train.py --train_log_dir ./result

For detailed usage instructions, see the Deep Learning Framework README.


🔧 Hardware Design

Fabrication of the NI-skin

The NI-skin is fabricated in three steps:

  1. Electrode array — Flexible high-density FPC with photolithographic patterning and chemical etching
  2. Ionogel sensing layer — Microstructured ionogel/TPU composite cured under UV
  3. Vacuum thermo-compression assembly — Monolithic integration of ionogel with electrode array using double-sided adhesive tape

This bilayer structure (coplanar electrodes + shared ionogel) eliminates interlayer alignment and enables high-density integration with minimal inactive sensing area.

Ionogel precursor composition
Component Amount
3,6-Dioxa-1,8-octanedithiol 2.7345 g (15 mmol)
1,4-Butanediol divinyl ether 2.9862 g (21 mmol)
1,1,1-Tris(3-mercaptopropionyloxymethyl)propane 1.5942 g (4 mmol)
Photoinitiator (DMPA) 0.036 g (0.5 wt%)
[EMIM][TFSI] 0.5852 g (8 wt%)

All components are thoroughly mixed until a clear homogeneous solution is obtained.

Interface Circuitry

The signal acquisition and processing pipeline implements the receptor and spinal layers:

Stage Component Specification
Excitation (DDS) AD9833 Frequency-tunable AC signals, 1–20 kHz
Analog Front-end OPA4388 High-pass filter (200 Hz) + voltage follower
C/V Conversion OPA4388 + TPL0501-100 Precision conversion with programmable adaptive gain
ADC AD7606 (16-bit) 51.2 kSPS, 8-channel synchronous sampling
Digital Processing Altera EP4CE10E22C8N FPGA Parallel FFT demodulation (512-pt) via DSP blocks + PZP crosstalk suppression
Data Bus USB / Ethernet Compressed data transmission to central processor (host PC)

PZP Crosstalk Suppression

A key innovation in the spinal layer is the parallel zero-potential (PZP) mechanism. All sensing electrodes are maintained at an equivalent zero potential, confining the excitation electric field to the addressed sensing node. This dual mechanism simultaneously suppresses:

  • Ionic crosstalk — prevents ion migration to surrounding sensing sites
  • Electrical crosstalk — interrupts loop current between parallel circuit elements

Crosstalk is reduced from ~39.4% to ~1.3%, outperforming even the ionogel-isolation scheme (~1.7%).


Datasets

This project includes three tactile datasets recorded at 100 Hz using NI-skin sensors.

1. Grasping Dataset — Object Recognition

Property Value
Task Object recognition during dexterous grasping
Classes 10 everyday objects
Repetitions 4 grasps per object
Duration 3–5 s per grasp
Sensor Hand NI-skin (48×48 elements, 69 units cm⁻²)
Robot Inspire-Robots RH56DFX-R dexterous hand

2. Touch Dataset — Gesture Recognition

Property Value
Task Gesture recognition on large-area skin
Classes 10 touch gestures
Subjects 4
Duration 3–10 s per gesture
Sensor Large-area NI-skin (96×96 elements, 30 units cm⁻²)
Robot Unitree Robotics G1 humanoid robot

3. Pain-Perception Dataset — Pain Estimation

Property Value
Task Pain level regression from nociceptive stimuli
Sharpness levels 5 (controlled by power-law index n)
Pain range 0–100%
Contact force ~500 mN (constant)
Duration 3–5 s per level
Sensor Fingertip NI-skin (24×24 elements, 156 units cm⁻²)
Robot Soft robotic finger on Universal Robot 10e

Deep Learning Framework

Our PyTorch-based cognitive layer supports three tactile perception tasks using 3D-DWT compressed sensing:

Task Model Input Output Performance
Object Recognition 3D-CNN (CNN_3D_Lite) Compressed tactile frames (CR=268) 10-class classification Accuracy >98.2%, DBI=1.17
Gesture Recognition 3D-CNN (CNN_3D_Lite) Compressed tactile frames (CR=268) 10-class classification Accuracy >95.4%, DBI=0.89
Pain Estimation Logit Regression Dual-sensor features (Pmax, Nact, ∇Pmax) Pain level 0–100% R² = 91.1%

Compression Pipeline: 3D-DWT → Quantization (Q factor) → Entropy encoding (zlib)

Lightweight Inference: Compression reduces network complexity from 3.95G FLOPS / 80K params → 938M FLOPS / 3.6K params (23.7% / 4.5% of original).

See the Deep Learning Framework README for complete training guides, Q-factor selection guidelines, and reproducibility recommendations.


Graphical User Interface

Two GUI applications built with PyQt5, PyQtGraph, and OpenCV for sensor debugging, data acquisition, and analysis:

  • 3D MAP Demo — Real-time 3D visualization of tactile pressure distribution across the sensor array
  • 2D Mapping — 2D tactile heatmap with data logging and replay

Both interfaces support: real-time data transmission, waveform visualization, graphical rendering, tactile map visualization, and data storage.


Dexterous Hand Control

Closed-loop tactile feedback control for embodied robotic systems:

  • Real-time sensor data reception via USB or TCP/UDP
  • On-device inference using trained deep learning models
  • Closed-loop control of Inspire-Robots RH56DFX-R dexterous hand and Universal Robot 10e arm
  • Communication via TCP/IP and ZeroMQ with Protobuf serialization

Demonstrated Capabilities

Task Robot Platform NI-skin Key Result
Fine texture perception Fingertip 96×96 (1,110 units cm⁻²) Resolves coin surface textures
Fine object manipulation RH56DFX-R hand 48×48 (69 units cm⁻²) Grasps 0.5 mm pencil lead, 0.2 mm solder wire
Large-area gesture sensing Unitree G1 humanoid 96×96 (30 units cm⁻²) Whole-body tactile perception
Nociceptive self-protection Soft finger + UR10e 24×24 (156 units cm⁻²) Detects needle puncture; triggers withdrawal reflex
Spiny object handling RH56DFX-R hand 48×48 (69 units cm⁻²) Reduces grip force upon pain detection


License

This project is released under the MIT License. You are free to use, modify, and distribute the code for both academic and commercial purposes.


Contact

This is an interdisciplinary project integrating material design, hardware development, and software engineering. We are actively improving the organization and documentation.

Corresponding Authors:

For code-related questions, please also contact Zhibin Li: 12231066@mail.sustech.edu.cn

We welcome your feedback and suggestions!


Acknowledgments

We thank all contributors and collaborators.

Author contributions: Z.L., B.D., and J.S. contributed equally. C.F.G., T.W., and C.S. supervised the project.


Made with ❤️ by the NI-skins Team · Southern University of Science and Technology

About

Toward Human-Level Tactile Perception: A Large-Scale, High-Density Iontronic Skin This project is developing a large-scale, high-density, and fast-response iontronic skin for human-level tactile perception in robotics.

Resources

Stars

24 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages