- Overview
- Key Features
- System Architecture
- Repository Structure
- Quick Start
- Hardware Design
- Datasets
- Deep Learning Framework
- Graphical User Interface
- Dexterous Hand Control
- Citation
- License
- Contact
- Acknowledgments
NI-skin is a large-scale neural-hierarchical iontronic skin system that mimics the hierarchical processing architecture of the human tactile sensory system. The human skin encodes and gathers signals from over 200,000 afferents with high spatiotemporal resolution — a capability that existing serial-architecture robotic skins cannot match when scaled to thousands of receptors due to readout latency, signal crosstalk, and bandwidth consumption.
Our system introduces a three-layer architecture — receptor, spinal, and cognitive layers — that processes tactile information through frequency-based position encoding, parallel transmission, backend convergent compression, and cognitive modeling:
- Receptor Layer — Ionotronic sensing elements position-encoded with optimized driving frequencies for parallel, crosstalk-free signal transmission
- Spinal Layer — Signal decoding, convergence, and 3D-DWT compression to minimize bandwidth
- Cognitive Layer — 3D-CNN-based spatiotemporal feature extraction for tactile recognition and pain perception
The system integrates >10,000 sensing elements with a minimum inter-sensor pitch of 300 μm (1,110 units cm⁻²) and a temporal resolution of 10 ms, enabling robots to perform fine tactile interactions and proactive injury avoidance through nociceptive sensing.
With a sensor density of 1,110 units cm⁻² — exceeding the mechanoreceptor density of the human fingertip (~240 units cm⁻²) — the NI-skin resolves fine surface textures of fabrics and coins at a feature size down to 300 μm.
The high-sensitivity, high-density NI-skin enables robots to delicately handle slender and fragile objects such as pencil leads (0.5 mm diameter), soft solder wires (0.2 mm diameter), and spiny chestnuts — tasks that demand fine tactile feedback and real-time pressure monitoring.
The NI-skin quantifies sharp-pain stimuli using a Logit regression model (R² = 91.1%) that fuses maximum pressure, activated-element count, and maximum pressure gradient. When the sensed pressure gradient exceeds the pain threshold, the robot triggers a protective withdrawal or grasp-release reflex to prevent skin damage — mimicking the human nociceptive response.
| Feature | Description |
|---|---|
| Neural-Hierarchical Architecture | Three-layer design (receptor → spinal → cognitive) emulating the human tactile nervous system for efficient parallel processing |
| 🔬 Ultra-High Sensor Density | Minimum 300 μm inter-sensor pitch (1,110 units cm⁻²), exceeding the mechanoreceptor density of the human fingertip (~240 units cm⁻²) |
| Parallel Frequency Encoding | 96 receptors per neural bundle, each encoded at 1–20 kHz with 200 Hz intervals; orthogonal frequency-division enables crosstalk-free parallel readout |
| PZP Crosstalk Suppression | Parallel zero-potential (PZP) mechanism reduces crosstalk from ~39.4% to ~1.3%, simultaneously suppressing both ionic and electrical interference |
| 3D-DWT Compressed Sensing | Three-dimensional discrete wavelet transform compression achieves CR = 268 (SSIM > 0.97), reducing bandwidth from 14.7 Mbps to 56.7 kbps |
| Pain Perception & Self-Protection | Logit regression-based pain model (R² = 91.1%) detects sharp stimuli and triggers protective reflexes for proactive injury avoidance |
| Scalable Multi-Scale Integration | NI-skins in multiple form factors (24×24 to 96×96 elements) deployed on fingertip, hand, and whole-body robotic surfaces |
| Real-Time Pipeline | End-to-end latency of ~23 ms (sampling → FFT → compression → transmission → inference), supporting closed-loop robotic control |
The NI-skin system emulates the hierarchical tactile processing of the human body across three functional layers:
┌─────────────────────────────────────────────────────────┐
│ COGNITIVE LAYER (Central Processor / GPU) │
│ 3D-CNN inference · Tactile recognition · Pain model │
│ Lightweight network: 938M FLOPS, 3.6K parameters │
├─────────────────────────────────────────────────────────┤
│ SPINAL LAYER (FPGA Edge Processor) │
│ FFT demodulation · PZP crosstalk suppression │
│ 3D-DWT compression (CR = 268) · Data aggregation │
├─────────────────────────────────────────────────────────┤
│ RECEPTOR LAYER (NI-skin Sensor Array) │
│ >10,000 iontronic sensors · Frequency-encoded (1–20 kHz)│
│ 300 μm pitch · 100 Hz readout · <600 μm thickness │
└─────────────────────────────────────────────────────────┘
| Metric | Value |
|---|---|
| Sensor density | 1,110 units cm⁻² |
| Total sensing elements | >10,000 |
| Minimum inter-sensor pitch | 300 μm |
| Temporal resolution (readout) | 10 ms (100 Hz) |
| Frequency-bin resolution | 100 Hz (512-pt FFT) |
| Adjacent leakage ratio (ALR) | <1% |
| Crosstalk (with PZP) | ~1.3% |
| Bending radius | 1.2 mm |
| Skin thickness | <600 μm |
| Operating pressure range | 0–100 kPa (linear), up to 550 kPa |
| Detection limit | <0.08 g |
| Two-point limen | 0.8 mm |
| Cycling stability | >100,000 cycles |
| Drift (100 kPa load) | ~9.8% |
| Compression ratio (optimal) | 268× (SSIM > 0.97) |
| Bandwidth reduction | 14.7 Mbps → 56.7 kbps |
| Grasp recognition accuracy | >98.2% (10 classes) |
| Gesture recognition accuracy | >95.4% (10 classes) |
| Pain model R² | 91.1% |
| End-to-end pipeline latency | ~23 ms |
NI-skins/
├── Hardware Design/ # Firmware, PCB schematics & circuit documentation
│ └── Circuit schematic diagram/
│ └── Schematic diagram.pdf
│
├── Graphical User Interface (GUI)/ # Python-based sensor visualization & data acquisition
│ ├── 1.Sensor array_3DMAP_Demo/ # 3D tactile map visualization
│ └── 2.Sensor_array_2Dmapping/ # 2D tactile mapping interface
│
├── Deep Learning Framework/ # PyTorch-based neural network models
│ ├── 1.TactileCompress_evaluation/ # 3D-DWT compression evaluation (CR vs. SSIM)
│ ├── 2.Object recognition/ # 3D-CNN for object classification (10 classes)
│ ├── 3.Gesture recognition/ # 3D-CNN for gesture classification (10 classes)
│ └── 4.Pain_model/ # Logit regression for pain estimation (5 levels)
│
├── Dexterous Hand Control Scripts/ # Real-time sensor→robot closed-loop control
│ └── 1.Soft robotic finger grasping/ # Gripper control via USB/TCP tactile feedback
│
├── assets/ # Images, GIFs, and figures
├── LICENSE # MIT License
└── README.md
- OS: Windows
- Python: 3.9+
- IDEs: STM32CubeMX / STM32CubeIDE / Keil5 / Quartus II / VS Code
pip install torch torchvision numpy matplotlib seaborn scikit-learn \
h5py pandas statsmodels tensorboardX tqdm PyWavelets thop-
Clone the repository
git clone https://github.com/jioy/NI-skins.git cd NI-skins -
Download datasets — unzip all data folders and place them under the corresponding
Alldata/directory. -
Update data paths — scripts contain default paths; update them to match your local machine before running.
Compression evaluation (sweep Q factors for CR–SSIM trade-off):
cd "Deep Learning Framework/1.TactileCompress_evaluation"
python Compression_evaluation.pyObject recognition training (3D-CNN, 10-class grasping):
cd "Deep Learning Framework/2.Object recognition"
python train.py --compress_q 2000 --train_log_dir ./resultGesture recognition training (3D-CNN, 10-class touch gestures):
cd "Deep Learning Framework/3.Gesture recognition"
python train.py --compress_q 2001 --train_log_dir ./resultPain estimation (Logit regression, 5 sharpness levels):
cd "Deep Learning Framework/4.Pain_model"
python train.py --train_log_dir ./resultFor detailed usage instructions, see the Deep Learning Framework README.
The NI-skin is fabricated in three steps:
- Electrode array — Flexible high-density FPC with photolithographic patterning and chemical etching
- Ionogel sensing layer — Microstructured ionogel/TPU composite cured under UV
- Vacuum thermo-compression assembly — Monolithic integration of ionogel with electrode array using double-sided adhesive tape
This bilayer structure (coplanar electrodes + shared ionogel) eliminates interlayer alignment and enables high-density integration with minimal inactive sensing area.
Ionogel precursor composition
| Component | Amount |
|---|---|
| 3,6-Dioxa-1,8-octanedithiol | 2.7345 g (15 mmol) |
| 1,4-Butanediol divinyl ether | 2.9862 g (21 mmol) |
| 1,1,1-Tris(3-mercaptopropionyloxymethyl)propane | 1.5942 g (4 mmol) |
| Photoinitiator (DMPA) | 0.036 g (0.5 wt%) |
| [EMIM][TFSI] | 0.5852 g (8 wt%) |
All components are thoroughly mixed until a clear homogeneous solution is obtained.
The signal acquisition and processing pipeline implements the receptor and spinal layers:
| Stage | Component | Specification |
|---|---|---|
| Excitation (DDS) | AD9833 | Frequency-tunable AC signals, 1–20 kHz |
| Analog Front-end | OPA4388 | High-pass filter (200 Hz) + voltage follower |
| C/V Conversion | OPA4388 + TPL0501-100 | Precision conversion with programmable adaptive gain |
| ADC | AD7606 (16-bit) | 51.2 kSPS, 8-channel synchronous sampling |
| Digital Processing | Altera EP4CE10E22C8N FPGA | Parallel FFT demodulation (512-pt) via DSP blocks + PZP crosstalk suppression |
| Data Bus | USB / Ethernet | Compressed data transmission to central processor (host PC) |
A key innovation in the spinal layer is the parallel zero-potential (PZP) mechanism. All sensing electrodes are maintained at an equivalent zero potential, confining the excitation electric field to the addressed sensing node. This dual mechanism simultaneously suppresses:
- Ionic crosstalk — prevents ion migration to surrounding sensing sites
- Electrical crosstalk — interrupts loop current between parallel circuit elements
Crosstalk is reduced from ~39.4% to ~1.3%, outperforming even the ionogel-isolation scheme (~1.7%).
This project includes three tactile datasets recorded at 100 Hz using NI-skin sensors.
| Property | Value |
|---|---|
| Task | Object recognition during dexterous grasping |
| Classes | 10 everyday objects |
| Repetitions | 4 grasps per object |
| Duration | 3–5 s per grasp |
| Sensor | Hand NI-skin (48×48 elements, 69 units cm⁻²) |
| Robot | Inspire-Robots RH56DFX-R dexterous hand |
| Property | Value |
|---|---|
| Task | Gesture recognition on large-area skin |
| Classes | 10 touch gestures |
| Subjects | 4 |
| Duration | 3–10 s per gesture |
| Sensor | Large-area NI-skin (96×96 elements, 30 units cm⁻²) |
| Robot | Unitree Robotics G1 humanoid robot |
| Property | Value |
|---|---|
| Task | Pain level regression from nociceptive stimuli |
| Sharpness levels | 5 (controlled by power-law index n) |
| Pain range | 0–100% |
| Contact force | ~500 mN (constant) |
| Duration | 3–5 s per level |
| Sensor | Fingertip NI-skin (24×24 elements, 156 units cm⁻²) |
| Robot | Soft robotic finger on Universal Robot 10e |
Our PyTorch-based cognitive layer supports three tactile perception tasks using 3D-DWT compressed sensing:
| Task | Model | Input | Output | Performance |
|---|---|---|---|---|
| Object Recognition | 3D-CNN (CNN_3D_Lite) | Compressed tactile frames (CR=268) | 10-class classification | Accuracy >98.2%, DBI=1.17 |
| Gesture Recognition | 3D-CNN (CNN_3D_Lite) | Compressed tactile frames (CR=268) | 10-class classification | Accuracy >95.4%, DBI=0.89 |
| Pain Estimation | Logit Regression | Dual-sensor features (Pmax, Nact, ∇Pmax) | Pain level 0–100% | R² = 91.1% |
Compression Pipeline: 3D-DWT → Quantization (Q factor) → Entropy encoding (zlib)
Lightweight Inference: Compression reduces network complexity from 3.95G FLOPS / 80K params → 938M FLOPS / 3.6K params (23.7% / 4.5% of original).
See the Deep Learning Framework README for complete training guides, Q-factor selection guidelines, and reproducibility recommendations.
Two GUI applications built with PyQt5, PyQtGraph, and OpenCV for sensor debugging, data acquisition, and analysis:
- 3D MAP Demo — Real-time 3D visualization of tactile pressure distribution across the sensor array
- 2D Mapping — 2D tactile heatmap with data logging and replay
Both interfaces support: real-time data transmission, waveform visualization, graphical rendering, tactile map visualization, and data storage.
Closed-loop tactile feedback control for embodied robotic systems:
- Real-time sensor data reception via USB or TCP/UDP
- On-device inference using trained deep learning models
- Closed-loop control of Inspire-Robots RH56DFX-R dexterous hand and Universal Robot 10e arm
- Communication via TCP/IP and ZeroMQ with Protobuf serialization
| Task | Robot Platform | NI-skin | Key Result |
|---|---|---|---|
| Fine texture perception | Fingertip | 96×96 (1,110 units cm⁻²) | Resolves coin surface textures |
| Fine object manipulation | RH56DFX-R hand | 48×48 (69 units cm⁻²) | Grasps 0.5 mm pencil lead, 0.2 mm solder wire |
| Large-area gesture sensing | Unitree G1 humanoid | 96×96 (30 units cm⁻²) | Whole-body tactile perception |
| Nociceptive self-protection | Soft finger + UR10e | 24×24 (156 units cm⁻²) | Detects needle puncture; triggers withdrawal reflex |
| Spiny object handling | RH56DFX-R hand | 48×48 (69 units cm⁻²) | Reduces grip force upon pain detection |
This project is released under the MIT License. You are free to use, modify, and distribute the code for both academic and commercial purposes.
This is an interdisciplinary project integrating material design, hardware development, and software engineering. We are actively improving the organization and documentation.
Corresponding Authors:
- Taihong Wang — wangth@sustech.edu.cn
- Chaoyang Song — Chaoyang.Song@mbzuai.ac.ae
- Chuan Fei Guo — guocf@sustech.edu.cn
For code-related questions, please also contact Zhibin Li: 12231066@mail.sustech.edu.cn
We welcome your feedback and suggestions!
We thank all contributors and collaborators.
Author contributions: Z.L., B.D., and J.S. contributed equally. C.F.G., T.W., and C.S. supervised the project.