PROJECT FINAL REPORT

Grant Agreement number: 323849

Project acronym: SimpleSkin
Project title: Cheap, textile based whole body sensor sensing system for interaction,
physiological monitoring and activity recognition
Funding Scheme: Seventh Framework Programme, Collaborative Project, ICT-2013.9.1
Period covered: from 2013-07-01 to 2016-06-30
Name of the scientific representative of the project's co-ordinator1, Title and Organisation:
Jun.-Prof. Dr. Jingyuan Cheng, German Research Center for Artificial Intelligence (DFKI)
Tel: ++49-(0)531-391-7469
Fax: ++49-(0)531-391-7445
E-mail: jingyuan.cheng@dfki.de

Project website address: http://www.simpleskin.eu

1
Usually the contact person of the coordinator as specified in Art. 8.1. of the Grant Agreement.

Page 1 of 61
4.1 Final Publishable Summary Report
4.1.1 Executive Summary
The aim of the SimpleSkin project was to develop basic technological advances needed to move smart
textiles and smart garments from the current status of a niche curiosity towards mass market applications.
The basic idea was to separate textile production, garment manufacturing, electronics development, and the
software implementation by well-defined abstractions and interfaces. Thus, instead of having to implement
an expensive special purpose solution for every application, “generic” mass producible components should
be available that can be flexibly put together to realize a variety of easily reconfigurable
applications[CLH2013,SHZ2015†].
Key advances towards this goal achieved in the project are:
1. Mass-producible, washable generic sensing fabric, which allows capacitive, resistive or impedance
modes, to measure movement, electrical body signals, activities, and change in body capacity[SEO2015,
VT2014]
.
2. Connector technology enabling arbitrary electronic modules (with a simple adapter) to be connected to
smart garment with tens of pins by merely being put into an elastic pocket[MVGT2015].
3. Techniques for producing various garments with flexible, application driven placement of various
sensing modalities from the above components.
4. Resistive, capacitive and inductive sensors utilizing on the generic fabric (see 1 above). This includes
detailed electrical and mechanical models, corresponding driving circuits, and detailed performance
evaluation[ZCS2014,HKC2014].
5. A signal processing and pattern recognition pathway for multiple modalities together with the
corresponding reconfigurable, scalable digital processing circuits and dedicated hardware
accelerators[VMV2015,ALC2016].
6. A user friendly Garment OS was realized on top of the Android platform based on a general architecture
and a self organizing application runtime environment with dynamic driver loader[SHB2014].
7. The above developments were demonstrated and evaluated in a variety of systems and applications
ranging from a smart soccer shoe[ZWM2016], through various muscle activity monitoring applications[ZSC2016,
ZSCXXXX]
to a multi-modal smart shirt focused on nutrition monitoring[CZK2013,ZFA2016].
In particular, the textile resistive pressure sensors which can be produced cheaply as a pressure sensor
matrix with up to 10,000 elements[CSZ2016] has proven to be attractive for a broad range of applications far
beyond wearable systems. This includes a smart table cloth[ZCL2015, ZCS2015], smart exercise mat[ZCS2014], and
smart seat covers for cars. The fact that the very same smart textile driven by the same electronics with
merely different software running on top can cover such a broad range of applications is a strong proof of
the SimpleSkin vision. The system has generated significant industrial interest with the car seat
demonstrators having been sponsored by Volkswagen, the smart shoe supported by Adidas and the general
wearable sports application being among the winners of the German Telekom “Fashion Fusion” contest‡.
The project has resulted in 42 peer reviewed scientific publications including top conferences and journals in
the field (ISWC, UbiComp, PerCom, IEEE Pervasive Computing, Elsevier Pervasive and Mobile Computing etc.)

†
The papers are indexed as in Appendix I: Publication and Internal Reports, where the list and full text of all peer-reviewed
publication are available.
‡
https://www.telekom.com/media/company/313834

Page 2 of 61
4.1.2 Summary Description of Project Context and Objectives
SimpleSkin project proposes a fundamentally new approach for creating smart textiles and
functional garments. The fundamental idea is to separate sensing textile production, garment
manufacturing, the hardware platform, and the software implementation by well-defined
abstractions and interfaces. A major innovation is the development of a mass-producible generic
sensing fabric, which will allow capacitive, resistive, bio-impedance modes, to measure movement,
electrical body signals, activities, and change in body capacity. The sensor density and intelligent
signal processing will compensate the simplicity of single sensors. Based on these fabrics “sensing-
ready” garments can be produced, that are with respect to their properties, looks, production
process and price virtually undistinguishable from today’s standard garments. We expect that in the
long term this will lead to functional clothes becoming the default, much like today smart, sensor-
enabled phones have become the mainstream. The “sensor-ready” garments become part of a
wearable computing system, by adding hardware, that allows self-organizing, dynamic and adaptive
processing of input signals converting the specific garment into a general wearable sensor with a
dedicated high-level sensing interface. By these means we create an abstraction layer and platform
on which application developers can create wearable sensing application, than are independent of
the actual hardware they run on. For example, this will allow an application developer to create a
sports monitoring application, that includes body posture, movement, and heard rate, which can be
deployed to any available “sensing-ready” shirt. This will empower a larger number of potential
developers to contribute their creativity. The approach taken in SimpleSkin has great potential to
build up the foundation for a new era in smart clothing. It aims at moving personal wearable
monitoring from a niche topic into major industry with the potential of revolutionizing what we
wear.

The components of the SimpleSkin modular Smart Textile vision: 1) Universal Sensing Fabric that is
a basis for a broad range of sensing modalities, 2) clothing made from such fabric where certain
broadly defined sensing structures are pre-prepared, 3) hardware platform that connects to the
clothing and turns the pre-prepared structures into concrete sensing modalities and 4) an OS
Platform on which concrete applications can be implemented as easy-to-write&use “Apps”.

Page 3 of 61
 The project aimed at breakthroughs on 4 layers:

Breakthrough 1. Sensing-ready garments as defaults, which enables the textile industry to

move from the ability to put a few special purpose textile sensors on a specific garment to the
ability to mass produce cheap, garment and application independent sensing fabrics. At the
core of this question are the tradeoffs and compatibility issues between production technology
concerns on hand and signal quality related physical requirements of the sensing on the other.
This also includes the limitations on the number of IOs between the garment and the
electronics as well as possible conflicting requirements of different sensing modalities, which
need to be placed on different layers. The breakthrough will result from a combination of
elaborate models of the sensing principles and an in-depth understanding of the textile and
garment production process and will be verified through demonstrations and simulations.

Breakthrough 2: Reliable and reconfigurable body and activity sensing. We aim to develop
methods and technologies for transforming the universal sensing fabric into a reliable and
reconfigurable source of complex multimodal information about the users' bodies and their
activities. The core breakthrough will be the ability to compensate the inherent signal quality
and reliability limitations of the universal sensing fabrics through appropriate processing circuits
and algorithms.

Breakthrough 3: Goal driven, adaptive, and self-organized sensing textiles. The control and
coordination logic poses a major challenge, because it faces a system that operates under
dynamic conditions with varying sensing requirements, varying information content of different
sensor locations and sensing modalities and varying sources of noise. We will address this
challenge through the development of a self-organized control methodology that uses
principles of attention to focus on sensor modalities and garment areas that are most relevant
in any given situation and can autonomously learn and evolve over time. The self-organization
will be driven by a description of the sensing goals, constraints, and priorities (e.g. accuracy of
certain information vs. power consumption) to be provided by the application.

Breakthrough 4: empowering developers and enabling a multitude of applications. The vision

of “wearable sensing as an App” aims to allow a broad community of developers to build
applications on top of garments made of the universal sensing fabric. The applications should be
independent of the specific garment and sensor configuration (as long as the garment can
provide the required information). Providing an abstraction layer for the creation of wearable
sensing applications is essential to make the implementation of such software economically
viable and attractive to a large number of developers. Much like the WWW enabled to creation
of distributed information systems by people with minimal programming skills and like the app-
store enabled a whole universe of phone applications, we aim at empowering a wide of
developers to create smart clothing applications.

Breakthrough 5: Demonstrating effectiveness. Evaluating and demonstrating the utility and

versatility of the sensing fabric allows a single specific garment to support vastly different
sensing application. We will demonstrate the benefits of the proposed concept in three highly
challenging and relevant application domains: activity recognition, nutrition monitoring, and
body driven human computer interaction.

To achieve and validate the above breakthroughs the project was organized around the 5 groups of
objectives and for each group a R&D work package was assigned.

Page 4 of 61
Objective Group 1 related to textile technology and the universal sensing fabric (WP1 Textile
Technologies). The goal was to create a cheap universal sensing fabric with the following
properties:

• high spatial resolution

• facilitating different sensing modalities such as capacitive, resistive, and inductive
concurrent multilayer integration of different sensing and communication structures
designed for mass production of fabric
• providing means for connecting large numbers of textile I/O to external devices
• decoupling the creation of sensing fabric and garment production
• allowing the selection of specific sensing configuration from a generic and universal sensing
fabric during garment production.

Objective Group 2 related to Sensing (WP2 Physical Sensing Layer). The goal was to transform
the universal sensing fabric into a reliable and reconfigurable source of complex multimodal
information about the users' bodies and their activities:

• developing models and simulations of the relationship between signals received from the
sensing structure in the garment and the human body
• deriving sensing principles and sensor architectures for detecting and measuring specific
parameters
• designing and developing analog electronic circuits for those sensing architectures

Objective Group 3 related to self configuration and data analysis (WP3 Goal Driven Adaptive
Processing). The goal was to create goal driven adaptive and self-organizing processing and
control architecture for universal fabric based sensing textiles.

• developing a set of goal oriented configuration, control and interpretation methods

• designing and implementing digital signal processing algorithms specifically adapted to the
envisioned sensing principle and tailored to the boundary conditions to the universal
sensing fabric.
• leveraging the understanding and models of the human body for efficient recognition of
basic actions and phenomena
• designing and implementing the envisioned processing architecture in a low power circuit

Objective Group 4 related to Garment OS (WP4 Garment Operating System). The goal was to
design and implement an operating system layer for garments that facilitates sensing as an app
through an application programming interface (API).

• creating an abstraction layer that allows software configurable sensor system configuration
• proving an API that allows the development of sensing apps that are independent of a
garment specific sensing configurations
• designing and implementing a self organizing application runtime environment that allows
multiple apps to make use of the garment as a sensing resource
• providing a user friendly development environment for textile sensing apps that enables the
development of such apps with programming knowledge only (no physics, signal processing,
or hardware knowledge require) based on building block approach

Page 5 of 61
 Objective Group 5 related to evaluation, demonstration and applications (WP5 Application
Development and Case Studies). The goal was to evaluate and demonstrate the utility and
versatility of the sensing fabric allowing a single specific garment to support vastly different
sensing application.
• Leverage the envisioned platform for novel forms of human activity recognition
• Extract, model and demonstrate new human computer interaction paradigms that are
enabled through sensing garments
• Develop and demonstrate new approaches to long-term precise nutrition monitoring
• Demonstrate the developers ability to envision and implement new application concepts
and specific sensing apps for sensing enabled garments
• Show in an integrated prototype applications of sensing garments that combines multiple
sensing apps in a single garment

Besides these scientific objectives, the project also strives to disseminate its results and pro-actively
monitor health risks and ethical issues as arising with the R&D tasks.

Page 6 of 61
4.1.3 Main S&T Results/Foregrounds
The result of S&T development will be grouped under 5 topics, corresponding to the 5 Objective groups and
5 Work Packages.

4.1.3.1 WP1 Textile Technologies

The Objectives are realized within 3 sub-work packages: 1.1 Multilayer Textile Sensing Structures; 1.2
Communication and Connectivity; 1.3 From Textiles to Garments.

WP 1 Highlights:
Ø Scaled-up production of multifunctional wash-stabile precision fabrics for all types of
sensing modalities
Ø pocket connector enables easy connecting between fabric sensors and electronics, the
concept is adaptable to different modalities
Ø Development, design and realization of demonstrator sensors and demonstrator garments
using the above mentioned fabrics and connectors
More Details: D1.1, D1.2, D1.3

. Universal fabric for multiple sensing modalities (supporting high spatial resolution)
Textile stands at the very beginning of the whole system. The development of universal fabric went through
several revisions throughout the project.
I. Survey on mass textile production technologies
Bearing mass-production possibility and production cost in mind right from the beginning, we specially
looked for technologies that already exist and with certain modifications can be used to produce the
universal fabric. A survey was first performed by textile partners (SEFAR and iTV) and the result is shared
with other partners, especially DFKI and ETHZ, which need to convert the fabrics into sensors and connectors
later, thus need the understandings on the possibilities and limits.
II. First textile prototype and further iterations
We selected weaving as the production technology because of its capability for high-throughput
manufacturing and the proven ability of the textile manufacturing partner Sefar for producing high precision
technical fabrics. Several different weaving patterns with conductive parallel strips were produced at the
beginning of the project using standard plain weaving processes. The Cu-Ag yarns were woven into the fabric
in order to obtain very dense conductive strips. The textile partners and sensor partners visited each other to
enhance understanding from both side (for example, the conductive stripes shall lie on one side of the
fabric) and the first prototypes were manufactured according to specification set and agreed by both sides.

Figure 1 The second iterations of universal fabric: fabric weaving pattern I with even separations (left),
and detailed view of the fabric showing the conductive stripes are located asymmetric (right)

Page 7 of 61
 III. Fabric sensor prototypes based on non-fabric prototype and further iterations
A sensing matric was first built by DFKI manually using commercial available material that meets the
electronic requirements. Several fabric based versions were then built taking this prototype as example. We
final chose the pure fabric solution because of the mechanic and electronic quality, stability during long-term
usage and the high manufacturing speed. These two types of fabrics are both based on weaving technology
and can be mass produced.

Figure 2 The hand-made non-fabric prototype(left)
and the 3 machine-made fabric-based prototypes with printing techniques, conductive foil pieces and the
pure fabric solution with “SEFAR Carbotex” (middle to right)
IV. Consideration on cost and washability
After we found the proper manufacturing methods and fixed the dimension specification, we tried to answer
two key questions if the fabric were going to be used in product, namely the cost and the washability. We
manufactured the fabric with two types of conductive material under the same dimension specification and
explored the influence on cost, comfort, flexibility, twistability, electronic performance. We then made
resistive sensing matrices out of both types of fabric and let them go through washability test, which
demonstrates the change of material properties after maximum 40 washing and try cycles using normal
washing powder and light-stain washing program in normal washing machine.

Figure 3 Washability test result: electronic performance of two types of material after multiple wash cycles
V. Supporting multiple sensing modalities
The fabric was initially designed for the resistive sensing und from the 2nd project years on, the capacitive
sensing and bio-impedance sensing were also integrated using the same fabric. A capacitive wristband was
created in the sensor shirt demonstrator in the 2nd project year. Bio-impedance sensors, which came into the
project later, were also created using the universal fabric in the final demonstrator (details in WP5 progress).

Page 8 of 61

Figure 4 the capacitive wristband within the sensor shirt with universal fabrics as capacitve sensors
Outcome: Scaled-up production of multifunctional wash-stabile precision fabrics for all types of
sensing modalities

. Means of connecting large numbers of textile I/Os to external devices

The sensors created using the universal fabric need to be interconnected to the driving electronics to make
them functional, here the challenge lies in how to route a large number of analog/digital IO to
interconnecting with the electronics and meanwhile ensuring features such as easy attaching/detaching of
electronics to/from the garment. A pocket connector is designed for such purpose. The idea is to have a
stretchable chest pocket with conductive connecting pads on the inside. The electronic device is then slid
into the pocket. Similar to a ball grid array chip, the device has small copper bumps in a matrix structure that
make contact to the conductive pads of the pocket.
I. From sensors to pocket connector
The resistive sensing poses the highest requirement on I/O number, embroidery is applied with specific
wiring scheme. The sensor nodes are wired by stitching the AWG36 wires on top of the demonstrator and
connecting these to the conductive weft by carefully soldering them to the fabric. The connection between
the pressure sensing pad and the pocket connector is also done by stitching the connecting wires on the
fabric. Specially to be mentioned is that the textile side of pocket connector is made out of the universal
fabric, too.

Figure 5 Stitching process of a sensor matrix
II. Pocket connector: from textile to electronics
The pocket connector consists of two parts. One part is the surface of an electronic device and the second
part is a stretchable textile in the form a pocket. When sized correctly, the stretched pocked should keep the
device in position and ensure also the electrical interconnection. The connecting surface of the electronic
device should have a slight curvature in order to optimize contact pressure at all positions.

Page 9 of 61

Figure 6 The Pocket connector: design concept (top left), simulated pressure distribution on casing with
bumps (top right) and real implementation (bottom)
Outcome: Easy connecting between fabric sensors and electronics, the concept is adaptable to
different modalities

. Selection of sensing configuration during garment production ( with decoupled fabric/garment production)
Given the methods to make individual sensor/sensor array/sensor matrix out of the universal fabric, and the
method to route large number of textile sensors to the electronics, the last step is to enable sensing
configuration during garment production. This was first tried out with smaller prototypes, where single
sensing modality was tested, and is best shown by the two demonstrator shirts we built in the 2nd and 3rd
project year, both include all three sensing modalities.
I. The 1st sensor shirt
With this prototype we tried for the first time integration of all three sensing modalities into one garment.
The resistive and capacitive sensors were manually cut out of the universal textile and the bio-impedance
sensor pads were still made of Shieldex conductive fabrics material, manufactured by Statex GmbH. This
shirt was then explored by USTUTT for various human computer interaction and activity recognition
purposes for WP5.

st
Figure 7 The 1 sensor shirt, the sensors are put out of their pockets

Page 10 of 61
 II. The 2nd sensor shirt (as final demonstrator)
Based on the experience gathered from the 1st sensor shirt, the final demonstrator shirt was built, where we
focused on the neck area and built all 3 types of sensors out of the universal fabric using a laser cutter.

Figure 8 From universal fabric to 3 types of fabric sensors: design in the final demonstrator shirt

nd
Figure 9 the 2 sensor shirt: sensors and their conductive yarn wiring under the collar (left) and sensor
outlook (right)
Outcome: Development, design and realization of demonstrator sensors and demonstrator
garments using the universal fabrics and pocket connector
4.1.3.2 WP2 Physical Sensing Layer
The Objectives are realized within 3 sub-work packages: 2.1 Modelling and Simulation for Sensing Structure;
2.2 Sensing Principles; 2.3 Analog Processing.

WP 2 Highlights:
Ø Modeling for capacitive and resistive sensing modalities
Ø Mechanical and electronic design space of sensors and their physical realization under
different application scenarios
Ø Mature hardware designs for multimodal sensing, that can be easily adapted to new
application areas with only small changes
Ø develop technologies that are more relevant to consumers derived from scientific results
More Details: D2.1, D2.2, D2.3

Page 11 of 61
. Deriving sensing principles and sensor architectures
The purpose of physical sensing layer is to convert the fabric into a media for information retrieval from the
human body. Three sensing modalities are chosen and developed, based on the amount of information that
can be retrieved, the complexity of sensor manufacturing, and the previous work from project partners.
I. Capacitive sensing
Using the user's body as dielectric of a capacitor, the change of the inner-structure of the body (such as joint
movement, muscle contraction, etc.) will reflect in the change of the existing capacitance between the skin
and the textile sensor. Therefore, such movements can easily be detected using capacitive sensing. The
major advantage of capacitive sensing is its high sensitivity, no direct body contact and relatively low
requirement of the sensing electrode. The challenge of the capacitive sensing includes the specific high
frequency that is required for the electric-magnetic field to permeate into the human body, parasitic
capacitance and shielding of the signal wire.
II. Resistive sensing
As most force-sensitive resistor (FSR) sensors, a sheet of flexible carbon-polymer material exhibits the
property that its resistance changes with mechanical deformation, e.g. when pressure is applied. In
SimpleSkin project, to implement a resistive-based force-mapping sensor, we use CarboTex from SEFAR as
the sensitive material and the metallic stripes which are woven into a normal polymer fabric sheet to
construct a matrix. We have focused on developing prototypes with a wide range of dimensions, from sport-
mat sized to elbow patch, of high sampling speed and resolution to evaluate a variety of ambient and
wearable applications.
III. Bio-impedance sensing
We introduced the measurement principles of biopotential and electrical bio-impedance. With 4-point
sensing, we can directly measure the electrical property of the target body tissues. The main challenge of
this sensing modality is particularly the impedance of the skin-electrode with dry textile electrodes.
Outcome: a selection of sensing principle suitable for large-scale textile implementation
. Models and simulations of the relationship between signals and the human body
Because bio-impedance sensing is already mature as the sensing principle, the models of the other two
sensing modalities are studied in more details.
I. Modelling capacitive sensing
Capacitive sensing uses two parallel electrodes to measure the change of the tissues under the electrodes'
coverage. Though capacitive sensing is a common solution in mobile, the relevant commercial touchscreen
solutions cannot be used to measure body tissue, mainly because: 1) the analog precision is typically not
enough for tissue activity measurement, 2) more importantly, the operating electro-magnetic field operates
at a lower frequency that it permeates mainly only into the air, so conductive objects such as a fingertip can
be registered; while for measuring body tissue movement, the filed needs to permeate into the tissue. We
carried out comprehensive study about the underlying principles. The result forms the guideline both for the
mechanical design of fabric sensor and interconnection, and for the analog driving circuit.

Figure 10 Capacitive sensing body model: energy distribution in human body

Page 12 of 61
 II. Modelling resistive sensing
Resistive-based force mapping measures the pressure force distribution on the sensing fabric itself. The
measurement of resistance is simply through a voltage divider, which has the features of fast response and
small physical component footprint. It therefore allows easy up-scaling of the matrix. However, when taking
a matrix of resistors, which are idealized as local resistances of the CarboTex material under the crossings of
electrodes, changing of a single resistor will influence the readings of the rest since it is an interconnected
matrix. For example, an actively pressed point will lead its entire row to have an offset, which is then
corrected on the software level. We have established a comprehensive study of the operation principles.

Figure 11 Resistive matrix simulation result with multiple points triggered, which enhances the
understanding of channel crosstalk thus contributes to noise suppression on software level
Also along with the development of the prototypes, several practical problems have occurred and renewed
our knowledge. For example, while powering up the active electrodes during the scanning, the not-powered
active electrodes can either be connected to ground or high impedance. Connecting to ground offers a
better mapping result; while in high impedance configuration, “ghosting” effect is more noticeable.
III. Modelling bio-impedance sensing
HB joined the project at project month 7 with already developed biopotential and bio-impedance hardware.
The focus is therefore adapting textile electrodes instead of commonly used medical gel electrodes, and
integrating the technology into wearable garments, that are linked to the work content in WP1, WP3 and
WP4. We performed a comprehensive test and proved that the use of the universal sensing fabric produced
by SEFAR, which is already used for resistive and capacitive sensing, is also feasible for bio-impedance
measurements.
Outcome: Mechanical and electronic design space of sensors and their physical realization under
different application scenarios.
. Analog electronic circuits for those sensing architectures
I. Capacitive sensing hardware
The detailed simulation demonstrated that a high operating frequency (near 20MHz), low noise level and
high dynamic range analog circuitry is needed to both cover both the large body movements in human
activities and also reveal movement details. We have developed several iterations of the sensing hardware.
They all support 4 channels measurement. While the critical components at the analog front-end did not
change much, the overall system evolved into a smaller, more energy efficient package.

Page 13 of 61
Version Month Descriptions and Major Changes
1.0 M6 4 channel capacitive sensing, 2 staged amplifier; analog, digital control, wireless
transmission (Bluetooth) on 3 separated boards
1.1 M18 1 staged amplifier, digitial control and wireless transmission (Zigbee) on the same board
2.0 M30 Compact, power efficient final design, analog and digital part on the same board,
supporting both BLE and WiFi communication

Figure 12 Analog circuitry of the wearable capacitive sensing

Figure 13 The evolution of capacitive sensing hardware into a smaller, more energy-efficient package
II. Resistive sensing hardware
For resistive sensing, two types of driving schemes are established for different applications, FPGA based for
high IO numbers and microcontroller based for compact-size and low power consumption.

Figure 14 Driving circuitry and system architecture of resistive sensing
The resistive sensing hardware evolved from the 1st proof-of-concept version with developing boards,
through 5 iterations to the self-designed, compact, energy-efficient end-designs.

Page 14 of 61
Version Month Resolution Descriptions and Major Changes
1.0 M6 128×128 First large-scale implementation; fast response analog switches to drive
active electrodes; fast analog de-multiplexer for channel switching before
high speed single channel ADCs
1.1 M12 120×60/120 Use smaller footprint connector (MiniDP), FPGA pins directly drive
electrodes.
2.0 M15 32×32 First portable implementation; duo 16-channel high speed ADC; analog
multiplexer to drive active electrodes - ghosting effect is obvious.
2.1 M16 32×32 Slim profile, FPGA directly drive active electrodes to rid of ghosting in 2.0;
USB-OTG enabled for smartphone based applications
3.0 M24 32×32 Low power microcontroller with integrated ADC; Li-Po battery powered;
Bluetooth Classic connection; very small footprint.

Figure 15 The evolution of resistive sensing hardware into a smaller, more energy efficient package
III. Bio-Impedance and biopotential sensing hardware
Biopotential measurement instrumentation uses a instrumentatio amplifier with very high input impedance.
The electrical coupling between the skin surface and the measurement electronics is done through the
textile electrodes. For electrical bio-impedance measurement, an electrical stimulus needs to be injected
into the tissue. Because of the stimulus injection, the interface between the dry electrodes and the skin
needs to have small impedance itself. Since the electrode polarization impedance is included in the actual
measurement result when, 2 pairs of electrode are used, the 4-electrode measurement technique is
required for characterizing the tissue impedance properly. Either a single frequency, or a combination of
frequencies can be used, enabling different applications.

Figure 16 Biopotential and Bio-Impedance Sensing Hardware and Schematics
In addition to the bio-impedance sensing front-end, a 1-channel biopotential amplifier is available for
implementing a 1-lead ECG recording.
Outcome: Mature hardware designs for multimodal sensing, that can be easily adapted to new
application areas with only small changes

Page 15 of 61
4.1.3.3 WP3 Goal Driven Adaptive Processing
The objectives are realized within 4 sub-work packages: 3.1 Adaptation and Self Organization; 3.2 Digital
Signal Processing; 3.3 Model Driven Pattern Analysis; 3.4 Low Power Implementation.

WP 3 Highlights:
Ø Configurable hardware structure supporting multiple sensing modalities
Ø General processing chain and pattern analysis combining different sensor modalities
Ø Low-level signal processing algorithm for each sensing modalities
Ø Result demonstrated through various of applications both as ambient and wearable
systems
Ø Hardware acceleration allowing scalability to bigger sensing matrices
Ø Selectable low power implementations and high-performance implementations
More Details: D3.1, D3.2, D3.3

Given textile sensors and analog front-end circuitry that provides raw signals, the next question is how to
reveal useful information directly related to basic body action through proper signal processing and organize
the hardware in a way that can be flexible configured.
. Set of goal oriented configuration, control and interpretation methods
we developed two reconfiguration and control schemes, to enable reconfiguration on physical layer and on
electronic layer.
I. Towards reconfigurable hardware: Building textile circuits using e-textile composites
We first considered the general configurability for embedding tiny component directly into textile. Due to
the desirable mechanical properties (e.g. bendability) of flexible electronics and sensors, there has been an
increased effort to bring them in the fields of smart textiles. We developed and demonstrated a novel
integration approach that creates a composite of flexible electronics and woven textile with conductive
fibres. The goal is to develop a programmable textile that can be used to configure a flexible system, e.g.
turn on/off sensors or dynamically route signals. The main components of the programmable textile (SRAM
cells and multiplexers) are fabricated on a 50μm thick freestanding polyimide foil using amorphous-Indium-
Gallium-Zinc-Oxide as a semiconductor and packaged in 7mmx7mm encapsulated electronic components.
These components are then interconnected using two layers of fabrics that form a grid of orthogonal 280μm
thick Ag-Cu fibres with a pitch of 750μm. This method can be expanded to also other small analog or digital
components and serve as a software configurable connection/bus structure for large-scale sensing matrix.

Figure 17 E-Textile Composite: concept and realization with flexible PCB and the universal fabric from WP1

Page 16 of 61
 II. Reconfigurable acquisition system for multimodal sensors
We have shown that different sensing modalities - the demonstrator used resistive and capacitive sensing -
can be combined into a single textile and signals can be separated using appropriate frequency filtering.
However, it was decided by the project partners to keep the sensing hardware separate, since for the
applications under investigation, different sensor modalities needed to be employed in different locations on
the body. The different modalities are then combined in software to enhance pattern analysis.
Outcome: Configurable hardware structure supporting multiple sensing modalities.
. Digital signal processing algorithms specifically adapted to the sensing principle and the boundary
conditions of the universal sensing fabric
I. General data processing chain
A general data processing chain for activity recognition using the raw digital data was established.

Figure 18 the data processing chain for resistive sensing represents the general data processing procedure
The chain contains 4 steps of processing, namely:
a. pre-processing: which shall move the DC/low-frequency drift component from signal, suppress noise
and artifacts specific for the sensing modality (e.g. channel crosstalk, ghosting effect).
b. Image analyse: this is for resistive sensing (one matrix per sample) only, which retrieves abstract, shift
and rotation invariant structural characteristics of the pressure distribution over the exact shape of
individual objects visible in the pressure distribution frame (e.g. overall weight, centre of weight,
contact area, number of contacts, Hu’s 7 moments). The output is a vector for each frame, similar to the
output of capacitive and bio-impedance sensing (several channel per sample).
c. Time-series analysis: temporal analysis is applied to multivariate time series, in a way similar to what is
usually done with IMU signals. Both time domain features (e.g. signal mean, standard deviation and
zero cross rate) and frequency domain features (e.g. main frequency, frequency centroid and energy)
can be used. When a specific action generates a distinguishable and repeatable pattern, template
matching becomes feasible. To handle actions at different speeds, dynamic time warping (DTW) is used.
d. Classification: the final classification is performed on a combination of spatial and temporal features.
Both the selection of the features and the specific classification algorithm are application dependent. In

Page 17 of 61
 most cases we have found that normalisation of the features using mean and standard deviation from
the training dataset can significantly improve the results.
II. Specific processing algorithms
Within the processing chain, proper algorithms for each step are carefully selected and/or developed, for
example: contact area extracting algorithm for feet detection and nutrition monitoring application, weighted
Dynamic Time Wrapping for sport detection, Differential Dynamic Time Wrapping for head gesture
recognition, ConfAdaBoost.M1 algorithm for busting fusion result using multiple classifiers.
III. Motion artifact compensation
We developed on the top of the processing chain an algorithm specifically taking motion artifacts into
consideration, which is very prevalent in smart textile application. Classical filtering approaches are often
inefficient when dealing with motion artifacts. Considering adaptive filtering, using an additional sensor as
artifact reference is often unfeasible in garments, e.g. placing an accelerometer in a neckband. Therefore, an
alternative approach is required for reducing motion artifacts in smart garments. We proposes a hierarchical
concept: the type of artifact (current state) is first detected, which is then used in the information extraction
step. This concept is suitable for different artifact sources affecting smart garments, e.g. heart rate as
physiological artifact, a certain way of garment displacement, or a certain degree of loosening of the
garment. This method is validated with capacitive sensing neckband and the result demonstrates am
enhanced recognition rate from the same dataset.

Figure 19 Concept of hierarchical artefact compensation
Outcome: General processing chain and pattern analysis combining different sensor modalities;
Low-level signal processing algorithm for each sensing modalities
. Efficient recognition of basic actions and phenomena of human body
Combining the above named technologies, we explored various application scenarios, both as ambient and
wearable system. Under most of the scenarios, we achieved efficient recognition rate of above 80%. The
applications explored the following areas:
• Capacitive: bio-parameters (breathing, pulse) monitoring, nutrition monitoring, head and wrist gesture
recognition
• Resistive: sport, daily activity recognition, group activity recognition, nutrition monitoring, user
identification, gesture recognition.
• Bio-impedance: bio-signal recording, nutrition monitoring
We also applied sensor fusion technics to get higher recognition rate in the nutrition monitoring based on
the final demonstration prototype (reported in WP1) using all 3 sensing modalities. More details will be
reported under WP5.
Outcome: Result demonstrated through various of applications both as ambient and wearable
systems
. Low power design and implementation
We investigate into the low-power design on three levels:
I. Choose of hardware

Page 18 of 61
This has been one of the major concerns while designing the ultraportable DAQ platforms. To address power
efficiency, we carefully examined the power consumption and selected low power consumption
components. Taking the capacitive sensing board V2.0 as example, we carried out the following measures:
a. MSP430-FR5739 as the microcontroller. It is of a popular ultralow-power microcontroller family. The
active mode (all clocks are enabled) consumes <1mA at 3.0V, and based on different applications, it has
several ultralow-power modes that gate off the subsystem that is not in use, which consumes <10uA;
b. The 24-bit high performance ADC supports both high-resolution mode (8.8mW, 2.9mA) and low power
mode(6.0mW, 2.0mA);
c. We dedicate power supply chips with digital enable on/off control to power the analog frontend,
therefore when the device goes to standby mode, the stimulus that is required by
capacitive/bioimpedance sensing can be turned off;
d. The overall current consumed by the data acquisition system is 7mA with ECG signal;
e. Bluetooth Low Energy as the data transmission solution, which consumes up to 15mA. The BLE protocol
can transmit 500 sample per second or 62sps when 8 ADC channels are all used. This is sufficient for
limited channel numbers as in capacitive and bio-impedance real-time measurement; however, for the
resistive matrix structure the bandwidth supports very limited area. To address this, we are currently
developing alternative module, which uses Bluetooth Classic protocol or operates in duo-mode.
f. High capacity Li-Po batteries (560mAh or 1300mAh) which can be found in most modern mobile devices
(smartphones and smartwatches) are used to provide power. Therefore, the device supports >25hr
battery life under continuous operation with the smaller battery.
II. Modulated design for switching between performance and power consumption
Wireless transmission is the part that consumes most energy in compact wearable platforms. We thus
separate the transmission module out from the data acquisition part. User can easily select between
performance and power consumption by physically plugging in the corresponding communication module
and selecting the software driver.
In general, the power consumption grows with data rate. We achieved in most cases a continuous running
time of above 8 hours for low data rate applications (capacitive/bio-impedance) and above 2 hours for high
data rate applications (resistive sensing), which is enough for working a whole day or for a session then
leaving the device for charging or switching battery.
Communication Data rate Power consumption Power through
Wireless, BLE Very low Very low battery
Wireless, Bluetooth low low battery
Wireless, WiFi High High battery
Wired, USB High High either directly through USB wire or separately
III. hardware based data process acceleration
To enable data processing on consumer devices (e.g. smart phone) and to consume as little energy and
processing resource on such devices as possible, we investigated into hardware-accelerated data processing.
We propose a FPGA based processing methodology, which not only accelerates sensing data processing but
also reduces the raw data size. The development time for FPGA designs is greatly reduced thanks to the
usage of an abstracted high-level synthesis approach. This system is validated using data from the resistive
sensing matrix but this strategy can be easily applied to other sensing arrays and grids.

Page 19 of 61

Figure 20 Hardware acceleration scheme: moving pre-processing and image analyse into FPGA

Figure 21 Data processing speed without (left) and with(right) FPGA acceleration demonstrates clearly the
advantage in the number of frames that can be processed real-timely per second
Outcome: Hardware acceleration allowing scalability to bigger sensing matrices and energy efficient
real-time processing on wearable devices; Selectable low power implementations and high-
performance implementations
4.1.3.4 WP4 Garment Operating System
The Objectives are realized within 3 sub-work packages: 4.1 Drivers and Abstractions; 4.2 Application
Programmer Interface and Templates; 4.3 Application Development Support.

WP 4 Highlights:
Ø Comprehensive understanding of the requirements on Garment OS as a middle layer
between the hardware platforms and the APP developers
Ø A general architecture allowing software configurable sensor system
Ø Self organizing application runtime environment with dynamic driver loader and easy
interface
Ø A user friendly Garment OS based on the Android platform was realized, thus enabling
application developers to reuse various existing features
Ø Evaluations performed in a multitude of applications as well as user tests
Ø Understandings on openness and privacy concerns of end users
More Details: D4.1, D4.2, D4.3, D5.2

Once hardware and customized data processing algorithms are ready, the next question is how to push the
technology developed in the project to a broader community, to free APP developers with programming

Page 20 of 61
knowledge only (no physics, signal processing, or hardware knowledge required). The Garment OS serves
here as the middle layer.
. Abstraction layer allowing software configurable sensor system
I. Requirements exploration
In the 1st project year, we worked towards the understanding on the Garment OS and made the first steps
towards it. The partners discussed how the different viewpoints can be unified to create a common
abstraction. Here we focused on the one hand on understanding the core requirements of the overall
SimpleSkin system and, on the other hand, on possible architectures for the overall system as well as
possible ways for utilizing specific parts such as the output channel. We organize a workshop on Smart
Garments at ISWC, the leading scientific conference on wearable computing. This event aims at bringing
together researchers and practitioners discussing different challenges and opportunities for integrating
sensors and actuators into garments as well as to identify application scenarios and explore different ways of
interacting with smart garments. We see this workshop and the prior engagement with the community as an
effective way of understanding the state of the art and the current research topics. Additionally, the
workshop contributed to a comprehensive understanding of the requirements for a generic API.

Figure 22 The main expertise necessary for developing a mobile system utilizing smart textile
II. Garment OS Architecture
Based on the knowledge gathered, we built the Garment OS final structure, mainly divided into three parts:
1) the Garment OS Service, 2) the Settings Application, and 3) the Garment OS SDK. The kernel part is the
Garment OS service. It handles the connection of sensors and actuators via Bluetooth and BLE, persists the
incoming sensor data, and allows sending data to actuators. The main user interaction takes place in the
Settings Application in which the user can connect sensors and actuators. The Settings Application connects
to the Garment OS SDK similar to applications developed by application developers via an API.

Page 21 of 61

Figure 23 Conceptual architecture of the Garment OS
Outcome: comprehensive understanding of the requirements on Garment OS as a middle layer
between the hardware platforms and the APP developers; A general architecture allowing software
configurable sensor system
. Self organizing application runtime environment that allows multiple apps to make use of the garment as a
sensing resource
I. Service layer for hardware managing
The main part of the Garment OS Service runs as a service in the background and handles the sensors,
persistence, communication and visualization. The output of this Service layer is packed and made available
to the user Apps.
a) Connectivity: communication between the Garment OS and the external sensors and actuators. Current
mobile phones offer a variety of different communication methods, most prominently Near Field
Communication (NFC), Wi-Fi, and Bluetooth. Each of these methods has its own specific advantages and
disadvantages. We have mainly focused on Bluetooth (using the Serial Port Profile) and BLE connections
as the default way of communication.
b) Persistence: The amount of data created using garment-based wearable sensors is huge. Thus,
persisting the data for long-term use as well as for analysis using larger time windows is mandatory. We
store all data generated by sensors first on the user’s mobile phone via text files. We implemented also
three cloud services, namely for Dropbox, OneDrive, and Google Drive. Before uploading the files to the
cloud service, they are packed and encrypted to ensure that the data is secure and the privacy of the
user is preserved.
c) Driver: Different garment-based sensors have different data formats. Thus, we developed a driver for
each sensor that interprets the received data stream and extracts the values. Further, each driver
implements a number of specific interfaces. These interfaces define which information can be extracted
from a sensor. In general, drivers are dynamically loaded at runtime. When the driver is successfully
loaded, the method is immediately invoked to potentially set up sensor properties. Then, the method is
loaded and passed to the thread that executes the method as soon as new data is received. The method
encodes the received data and extracts information. The drivers used for the actuators send strings to
the actuators. These strings can be simple strings (e.g., “heart rate 90”) or serialized matrices of vectors
(e.g., for LED matrix). The simple strings are pre-defined within the Garment OS.

Page 22 of 61

Figure 24 Drivers are dynamically loaded at runtime
II. Settings App enables easy hardware configuration
In order to enable the end user to easily control the Garment OS, we developed an Android application. The
user can manage preferences such as privacy, persistence, or the used sensors and actuators. The Settings
Application uses a specific API that has additional functionality compared to the regular application API such
as management functions for the sensors and actuators.
a) Manage Sensors and Actuators: The view presents a list of available devices. The user can add new
devices, enable existing devices, remove existing devices, or view device details. The device detail view
presents information about the device and visualizes the current sensor values.
b) Manage Privacy: the user can select which application is allowed to access certain devices. The user
chooses the device based on the information level requested by an application. This also makes sure
that the user has all necessary sensors and actuators for an application.
c) Manage Storage: the user can upload the measured sensor data to cloud services (with encryption
enabled or disabled) or save them to the local file system.

Figure 25 Different views of the settings application of the Garment OS:
The main menu (left), sensors view (middle) and the sensor information view (right)
Outcome: Self organizing application runtime environment with dynamic driver loader and easy
interface for that
. API allowing the development of sensing apps independent of a garment specific sensing configurations
The SDK provides an API to connect applications to events (i.e., using call-backs), to get the current value of
sensors (i.e., polling), or to send data to actuators. Further, it includes a developer module to support novice
developers. We developed an API that can be used by the application developer to create applications for

Page 23 of 61
the garment OS. While these functions cover a large set of typical sensors and actuators, specific sensors
might need different functions and, thus, the API could need to be extended in future. In general, the API is
divided into two parts: one consisting of calls that return the sensor and actuator objects themselves, this
way developers can use the sensor objects to access their values and the actuator objects to send
information to them; the other has predefined functions for returning popular information such as heart rate
and step count.
We implemented a cross-platform software stack that applications written for different SimpleSkin devices
can run on. The engine provides several key features:
• Command line interface
• Live plotting / animation of incoming sensor data
• Raw data recording and reprocessing (playback)
• Data recording as CSV for export/offline processing
• Streaming data via TCP to a network connected device for online/offline processing
• Cross-platform interoperability

Figure 26 Excerpt of the API from the Android AIDL
Outcome: A Garment OS based on the Android platform was realized, thus enabling application
developers to reuse various existing features

Page 24 of 61
. User friendly development environment for textile sensing apps based on building block approach
We have always kept user friendliness in mind when developing and implementing the above methods.
In the first 2 years, we proposed and evaluated an initial set of methods of Garment OS on simple prototypes
with one sensing modalities. In the last year, we extended the set of applicable algorithms not only for single
sensing modality application, but with all 3 sensing modalities. For example, we used the final demonstration
shirt with all 3 sensing modalities and validated the development of nutrition monitoring APPs based on the
Garment OS.
On the top of validation on SimpleSkin hardware prototypes, we dressed also the necessity of exploring the
user’s knowledge of what can be actually inferred by sharing personal data. We performed first a literature
review on wearable sensors and the information that can be derived from them, then carried out an online
survey to assess users’ willingness to share information from their wearables. Our survey was completed by
249 participants (127 male, 115 female, 7 did not specify). We looked into three aspects: willingness to
share, sharing raw sensor data vs. Derived information, to whom the information can be shared.

Figure 27 One of the results from the on-line survey demonstrates to what degree users would like to share
different types of data and information
Outcome: user friendly Garment OS validated by two applications and enhanced by understandings
on openness and privacy concerns
4.1.3.5 WP5 Application Development and Case Studies
The Objectives are realized within 4 sub-workpackages: 5.1 Case Study - Adaptive Activity Recognition; 5.2
Case Study – Long-term precise nutrition Monitoring; 5.3 Case Study - Versatile Generic Human Computer
Interface; 5.4 Open Application Challenge.

WP 5 Highlights:
Ø Adaptive activity recognition using the resistive pressure matrix
Ø Realization of nutrition monitoring using multiple SimpleSkin sensors
Ø Different interaction techniques for capacitive and resistive sensors realizing various
application scenarios
Ø Integrated prototypes with multiple sensing apps in a single garment
Ø Workshops with hand-on sessions to realize certain applications
Ø Seminar developed for students with ready-to-use hardware and data mining library
More Details: D5.1, D5.2

Page 25 of 61
With the universal fabrics (WP1), the sensing platforms (WP2), the data processing technologies (WP3) and
the Garment OS (WP4) as a middleware, in WP5 we demonstrated with a large variation of applications our
proposed smart textile system in activity recognition, in nutrition monitoring, in human computer
interaction. We also validated the open application challenges through workshops with researchers and
prepared seminar for students who will be the future researchers and work forces on smart textile.
. Leverage the envisioned platform for novel forms of human activity recognition
Though the project years we validated the following human activity recognition with the smart garment, an
overview of explored applications are list below.
Sensing Physical principle Context recognized Potential
Technology domain
Wearable muscle activity monitoring in gym exercise
Resistive Surface pressure changes Muscle behaviour: Type and repeat Sport,
matrix between the skin and an of exercises, workout quality (force healthcare
elastic sport support band pattern variation and consistency)

SmartShoe for football kicking monitoring
Resistive Surface pressure changes on Type and direction of kicking, the Sport, sport
matrix the shoe when shooting the pressure change on fabric shoes
ball manufacturer

Smart-Mat for gym exercises evaluation

Resistive Surface pressure changes Type and repeat of exercises, Sport,
matrix between the body and sport workout quality (balance, speed, healthcare
mat consistency)
Smart-CarSeat for monitoring the driver's activity
Resistive Surface pressure changes Driver body posture and activity, Car industry,
matrix between the body and the identity, active level security
seat
Smart-Floor for in-door localization
Resistive Surface pressure changes People’s identity and location, Location-
matrix between the feet and the furniture’s location based
ground service,
security
Smart-Floor for upper body activity recognition
Resistive Surface pressure changes Interaction of a person’s upper body Work
matrix between the feet and the with furniture support,
ground security

Outcome: Adaptive activity recognition using the resistive pressure matrix

. Develop and demonstrate new approaches to long-term precise nutrition monitoring
We started nutrition monitoring with capacitive sensing and expanded step by step to the final
demonstrator, where all 3 sensing modalities are used.
I. Nutrition monitoring with capacitive neckband
Capacitive neckband was developed with 4 channels to detect the structural change of the neck, that is
directly related to swallow. We demonstrated first in a controlled lab environment, how head and neck
related events (e.g. swallowing, head motion) can be detected. We then moved to data recording during
every day activities and demonstrated that while the detection of individual swallows and head motion may
work poorly in real life data streams, a statistical distribution of swallow frequency, time between swallows
and head motion can be detected reliably enough to be a good indication of certain activities. In a data set of
138 hours recorded from 3 participants we are able to detect 24 out of 25 big (breakfast, lunch, dinner)
meals with just 3 false positives. When smaller meals (e.g. eat a banana) are included, we detect 46 out of 64

Page 26 of 61
with 136 false positives. Beyond eating, the next interesting result is the ability to distinguish between just
being absolutely quiet (no motion) and actually sleeping, as periodic “empty swallows” occur more seldom
when a person is sleeping.
II. Nutrition monitoring with resistive table-cloth
We investigated the pressure distribution on the surface underneath the plate from which the food is eaten.
The core idea is that such pressure information can also be used to distinguish various cutlery related
activities such as cutting, poking, stirring or scooping. We showed how to spot such individual actions in a
continuous data stream, assign them to specific containers (main plate, salad bowl), count them (e.g. how
many bites were taken), and relate them to different abstract categories of food. We presented the results
of a comprehensive study with 10 participants, each having consumed a total of 8 meals chosen from 17
possible main dishes (divided into four categories according to the predominant cutlery action involved)
combined with 6 possible side dishes (divided into 2 categories).
III. Nutrition monitoring with the final demonstrator shirt with all 3 sensing modalities
As the final demonstrator, which shall combine and present progresses of all WPs, we integrated low-profile
collars on upper-body shirts with three types of sensors: capacitance sensor, bio-impedance sensor, and
resistance based pressure sensor, and used this shirt for nutrition monitoring.

Figure 28 Three sensors made of the same textile:
A) capacitance sensor plate, B) bio-impedance electrodes, C) resistance sensor patch.

Figure 29 The shirt collar with 3 sensing modalities used for nutrition monitoring:
A) inner surface of the collar, B) outer surface of the collar, C) measurement scenario with the shirt
We used the bio-impedance data and the resistance data recorded from 3 out of the 10 participants under
laboratory condition in a pilot study to detect swallows. Afterwards, we extended the study to all the 10
participants and merged data of the three sensor types for swallow detection. Averagely 93% of the
swallows were detected and with a precision of 85%. Further analysis of the full 10 participant dataset in
laboratory and free-living conditions are on-going. In particular, promising results for the fusion of the three
sensor sources were obtained and shall be extended in further work.

Page 27 of 61
 Figure 30 Example swallowing signals: from top to bottom, capacitance channel 1, capacitance channel 2,
resistance, and bio-impedance. The swallows are labelled out with purple segments
Outcome: Realization of nutrition monitoring using multiple SimpleSkin sensors
. Extract, model and demonstrate new human computer interaction paradigms enabled through sensing
garments
Though the project years we validated the following human computer interaction paradigms with the smart
garment, the overview of explored applications is given below.
Sensing
Taxonomy Description Picture
Technology
Navigation: cruise control for pedestrians
Electric muscle Contextual input @ As a new kind of pedestrian navigation paradigm that
stimulation external system à primarily addresses the human motor system rather
(EMS) physical output @ than cognition, we propose the concept of actuated
lower body legs navigation. Instead of delivering navigation
information, we provide an actuation signal using EMS
that is processed directly by the human locomotion
system and affects a change of direction.
Controlling smart watches using fabric-based sensors
Capacitive Physical input @ We propose using capacitive sensors integrated into
wristband upper body hands the watchstrap to enable single hand and hand-free
à visual output @ interaction. We elicited gestures to control the watch
upper body arms by conducting a user-defined gesture study.
Hands-free gesture control with a capacitive textile neckband
Capacitive Physical input @ We present the neckband for hands-free gesture
neckband neck à visual controlled user interfaces allowing continuous
output @ external unobtrusive head movement monitoring. We explore
system the capability of the proposed system for recognising

head gestures and postures.
Automotive domain applications
Bio-impedance Physiological input We propose detecting the physiological state of the
sensors @ upper body torso driver via bio-impedance sensors using signals such as
à contextual heart and breathing rate. These vitals can be
output @ external continuously monitored and the driver can be alerted
system to stop or take a break when there are any problems
or a high workload.
Implicit engagement detection for interactive museums using brain-computer interfaces

Page 28 of 61
EEG Felt-based Physiological input We provide a rich and more personalized museum
sensors @ head à visual experience using implicit input from Brain Computer
output @ systems Interfaces (BCI). EEG signals are used to detect visitor
engagement in exhibits, which is then used to build a
recommender system that suggests similar exhibits or
routes that the visitor could take for a personalized
and more engaging experience.
Fabric-based touch input
Resistive matrix Touch input on- Touch sensitive fabrics allow making each piece of
body or ambient à clothing to be fully touch enabled similar to the
control information displays of these devices. Thus, the input that is
for external system currently performed on the mobile device can also be
performed on clothing.
Visual Output
On-body display Visual output We explore at which location potential users prefer on-
prototype body displays for either personal or public usage. In
addition, we explore different visualizations for each of
the application scenarios.
Interaction with public displays using smart garments
Resistive Physical input @ We mimic the input that is nowadays sensed by depth
sensors, Bend upper body torso à cameras by using sensor patches at the user’s joints.

sensors visual output @ We first compare both sensors with baselines such as
external system the Microsoft Kinect then explore different interaction
techniques including gesture input and pointing input.
Secure authentication utilizing EMS
Soft fabric Contextual input @ We propose utilizing EMS as a mean to send cues
Electric Muscle external system à between the user and the system that only the user
Stimulation physical output @ perceives without producing acoustic or visual
(EMS) upper body arms feedback. We use EMS to stimulate the user’s muscles
electrodes in certain patterns, which indicate a number that can
be added to the current PIN input.
Textile-based brain computer interface for mobile interactions
Conductive Input @ upper body we propose using Brain Computer Interfaces (BCI) in
textile head à visual everyday life for mobile interaction, using textile
electrodes output @ external electrodes developed within SimpleSkin to design a
system head cap (winter cap/sport cap) that can detect EEG
from the brain as well as EMG and EOG from the facial
muscles and the eyes.

Outcome: Different interaction techniques for capacitive and resistive sensors realizing various
application scenarios
. Show in an integrated prototype applications combing multiple sensing apps in a single garment
The integration is best shown by the two sensor shirts we developed in the 2nd and 3rd project year. Both
combine all 3 sensing modalities (the 1st shirt used partially the universal fabric, while the 2nd makes full
usage of the universal fabric for all 3 sensing modalities). Based on these two shirts multiple sensing
applications were explored. Most of the above named applications in human activity recognition, nutrition
monitoring and human computer interaction, have been validated using these two shirts or using
technologies that can be easily integrated onto the shirts.
Outcome: integrated prototypes with multiple sensing apps in a single garment
. Demonstrate the developers ability to envision and implement new application concepts and specific
sensing apps for sensing enabled garments
The developers ability to envision and implement new application is greatly shown by the large number of
applications we explored within only 3 years (as listed above). These were majored done by researchers and

Page 29 of 61
the students working for the project. The work were carried out by people on all level, not only by Ph.D.
candidates who fully engaged themselves onto the topic, but also by master students who dug into one
specific application within only 6 months. At the latest stage of the project, master students (student
assistants working on 10 hours/week base) were also able to develop SimpleSkin systems utilizing ready
hardware design and software algorithms, developed by the experience researchers, and create their own
design and applications.
Beyond project members, we aim at enabling more generic community to easily develop smart textile
application.
For that purpose we started in 2015 investing into the application challenge, which shall provide a lot of
insights and help for the system development. This first version of the application challenge will help us to:
1) increase the usability, 2) find bugs and missing parts, and 3) get insights into how to conduct application
challenge events. Therefore, we conducted a workshop at Mobile HCI conference 2015 that focuses on
creating applications for wearable devices (http://blog.hcilab.org/frommobiletowearable). We chose this
venue because of the wide focus of mobile interaction since we hope to get new ideas that can be realized
with the SimpleSkin system. The main part of this workshop was a hands-on session in which participants
developed application based on the garment OS developed in WP4. We provided a number of garment-
based sensors and actuators mostly developed within this project. However, we do not want to limit the
participants to garment based sensors and actuators and also provided other such as accelerometer,
displays, or simple buttons.
Heading the end of the project, we worked towards an open-access developer kit. A screenshot for the
library is given below and more detail can be found in section 4.2, part B2, “General textile sensing platform
for research community”.

Figure 31 On-line documentation on application workflow and help files for a ready-to-use library
Outcome: Workshop with hand-on sessions to realize certain applications; Seminar developed for
students with ready-to-use hardware and data mining library

Page 30 of 61
4.1.4 The potential impact
Highlights:
Ø 42 scientific publications
Ø 1 Ph. D. thesis, 17 bachelor/master theses
Ø 2 industry projects utilizing SimpleSkin results
Ø 1 long-term partnership with Volkswagen DataLab
Ø 1 start-up in plan, with knowledge gathered in Lean Launchpad Pilot program and plan for
proposal to FET Innovation Launchpad
Ø demos to general public at Cebit’15,16, Girl’s day and etc.
Ø news coverage from Bayerischer Rundfunk, Rhein-Zeitung, DW.de and many more
More Details: D6.1, D6.2. D6.3

4.1.4.1 Socio-Economic Impact

. Industrial Cooperation:
Within the 3 project years, the partners have successfully attracted attention from industry and realized
technology-transfer through 4 industry projects. These projects demonstrate the initial application of
SimpleSkin project in a broader area. 3 of these projects are with car industry, 1 project is with measurement
instruments manufacturer. All 3 cooperation between DFKI with industry were realized in the format of
technology-transfer, where one prototype with SimpleSkin sensing modalities for a concrete application was
developed for the industry partner. The 1st industry project begins already in the first project year and
locates mainly in the 2nd and 3rd project years.
• Project 1: Smart Car Seat (Term I)
Industry partner: Volkswagen DataLab
Period: 05-12.2014
Format: technology transfer
Content: DFKI takes the general pressure sensitive fabric technology developed in SimpleSkin to develop the
prototype of a smart car seat, which is able to detect driver’s body posture and activity level. The system
shall help the smart car react more properly according to the driver’s status (e.g. slow down when the driver
turns around to car the baby).
Result: the prototype and the program was successfully delivered. Following project and long-term
cooperation merged based on this cooperation.
• Project 2: Smart Car Seat (Term II)
Industry partner: Volkswagen DataLab
Period: 11-12.2015
Format: technology transfer
Content: DFKI continues the work on car seat with pressure sensitive fabric, and develop the algorithm and
user interface to detect the user identity (who is sitting on the car seat). The system shall provide an
unobtrusive method for 1) personalized service in car; 2) additional security check on the driver.
Result: the prototype and the program was successfully delivered.
• Project 3: Demonstrator Future Truck Seat with Smart Textile
Industry partner: Johnson Control Components GmbH & Co. KG
Period: 04.2015-03.2016
Format: technology transfer

Page 31 of 61
Content: DFKI implement a ready-to-use prototype with textile resistive sensing for body pressure
distribution and as control panel, textile capacitive sensing for respiration detection, textile display for
information feedback. The prototype shall be demonstrated on a workshop or at the company and pictures
and videos shall be provided to the company for future use.
Result: the prototype was successfully demonstrated.

Figure 32 System Setup for Project 3 Demonstrator Future Truck Seat
• Project 4: Sensor analysis
Industry partner: Thyracont Instruments
Content: Sensor analysis project sponsored by company was implemented and concluded.
Result: further project under planned.
• Long-term partnership with DataLab Volkswagen
The cooperation between DFKI and DataLab Volkswagen starts from the Smart Car Seat project, which is a
direct result from SimpleSkin project. The long-term partnership was fixed in Jan. 2015. The innovation IT-
solutions of “Big Data” and “Internet of Things” are to be developed with this close cooperative partnership.
Details can be found on DFKI website: http://goo.gl/adbfCr
. Ethical Impact:
On one hand, the involvement of human subjects in SimpleSkin project is prevalent, on the other hand, most
of the involvements stay on a small scale and the experiments are designed as close to natural life as
possible, because it is also the interest of researchers in the field of Human Computer Interaction and
Ubiquitous Computing to have the data recording scenario as similar as possible (if not directly from) to
natural life and to project as little burden as possible to the subjects. The worst feedback reported by the
subjects are like: the experiment was tiring (in sport monitoring), or, I ate a little bit too much (in nutrition
monitoring).
We have followed the practical framework for Ethics of project pd-net (http://pd-net.org/ethics/), a FP7
project where Uni. Stuttgart participated, as discussed and decided in the project meetings of the 1st year.
The following templates were created and made accessible to general public on SimpleSkin website
(http://simpleskin.org/?ethics). The full content can be found in Appendix II to the deliverables.
• Procedures for Volunteer Studies
• Procedures for Public Trials
• SimpleSkin Ethics Primer
• SimpleSkin Ethical Worksheet
• SimpleSkin Project Overview (to be handed out to experiment volunteers)
• Informed Consent
• Guide to Secure Data Storage

Page 32 of 61
4.1.4.2 Dissemination Activities and Exploitation Results
. Scientific publications
SimpleSkin has achieved outstanding publication success and has far exceeded the dissemination targets
initially set. In total we published 42 peer-reviewed papers including top. We actively participated create
community of researchers by organizing workshops and promoting research on smart textile at high ranked
conferences. The full publication list and the full content can be found in Appendix I to the deliverables.

Publication number per year

20

15
book chapter
10
report
5
proceeding
0
2013 2014 2015 2016 under journal
review/ to
be
submitted

Figure 33 Publication amount overview
On average 2.2 publications per partner per year is achieved.
The majority of publications were published in conference proceedings, which is common in the field of
Computer Science and for some of conferences where we published, the acceptance rates are as low as 10%-
25% (UbiComp, CHI, ISWC, PerCom). The number of publications is stable through the 3 year, with a burst of
accepted and to be submitted papers at the end of the project (4 in the 6 months of 2013, 15 in 2014, 11 in
2015, 16 in the 6 months of 2016, and 6 under review or to be submitted).
The electronic copies of all these publications can be found both in Appendix I and on the SimpleSkin
Website. The papers accepted at conferences were or are going to be presented to the research
communities as talks and/or posters, demos.

Figure 34 Full publication list and open access to all papers (http://simpleskin.org/?publications)

Page 33 of 61
At the beginning of SimpleSkin, all partners cooperated on an outline paper, which was published on IEEE
Pervasive Computing and aiming at disseminate the project and its methods to the community. At the end of
the project, Uni. PASSAU(Oliver Amft) and USTUTT(Stefan Schneegass) co-organized one book on Smart
Textile and Textile Electronics, aiming at providing a holistic view on the design and development of smart
textiles and textile electronics.

Figure 35 Co-publication at IEEE Pervasive Computing

Handbook on Smart Textiles and Textile Electronics
Publisher: Springer, HCI Series
Appear: End of 2016
Key topics: Holistic view on Smart Textiles including textile production, sensor/actuator design,
interaction design, application scenarios, and more.
SimpleSkin partners’ contribution: Introduction on Smart Textiles (USTUTT, Uni. PASSAU), Textile
Production (iTV, SEFAR), Pressure Force Mapping (DFKI), Reversible Contacting for Smart Textiles
(ETH), Textile Antennas (ETH), Electronics integration (ETH), SimpleSkin Approach: Decoupling
Textiles, Sensing, and Applications (all)
External contribution: Keio University, University of Pisa, Department of Textile Engineering Namik
Kemal University, UdK, University of Hannover, Bauhaus University Weimar, University of Bremen,
University of Lapland, University Nürnberg-Erlangen
Summary: The Handbook of Smart Textiles and Textiles Electronics provides a holistic view on
the design and development of smart textiles and textile electronics. Currently, researchers and
developers mainly focus on a single aspect in the production process of smart textiles without taking
requirements posed by other disciplines into account. With leading experts in different domains
from textile production, electrical engineering, interaction design, and human-computer interaction,
this handbook focuses on the whole production process. It discusses the most important aspects
ranging from textile manufacturing, sensor and actuator development for textiles, and integration of
electronics into textiles to interaction with textiles and different application scenarios realizable with
smart textiles. The handbook is ideal for researchers, designers, and academics who are interested in
understanding the overall production process of smart textiles. It is particularly well suited to get an
introduction to the different fields involved in creating smart textiles and it lays a basic
understanding of the steps necessary to produce a successful smart textile.
. Creating Community
We not only strived to push our research to existing research community (in for example human computer
communication, wearable computing, ubiquitous computing), but also worked hard to create and enhance
sub-community especially for smart textile.

Page 34 of 61
 • Uni. Passau co-organized with ETHZ the conference BSN 2014 and served as technical program chair,
this event provided a platform to present on-body textile technology and applications.
• Uni. Passau organized the International Workshop on Sensor-based Activity Recognition and
Interaction(iWOAR 2016, Program Co-chair) and International Conference on Wearable and
Implantable Body Sensor Networks (BSN’ 16, Steering committee member)
• USTUTT organized a workshop on enriching mobile input with wearable devices in August 2015. In
this workshop we strive to generate further use cases and discuss with experts for mobile interaction
the possibilities to use smart garments and other wearable devices on a daily basis. The workshop
will be held at the MobileHCI conference that is the main event for researchers working on new
interaction methods for mobile devices.
• USTUTT, DFKI and Uni. Passau organized workshop on Smart Garments at the International
Symposium on Wearable Computing 2014 (held within the 2nd project year but reported already in
D4.1 and D6.1).
• DFKI organized with University of Science and Technology of China the 1st Sino-German Symposium
on Social Interactive Computing, funded by German DFG and Chinese NSFC, hosted at DFKI, April.
2014, where the result of SimpleSkin is presented to 30 German and Chinese group leaders
(professors and post-docs).
• DFKI will organize and host UbiComp/ISWC’16, one of the highest ranked conferences in the
ubiquitous computing community (co-chair, demo-chair, local organizing chair etc.), USTUTT and Uni.
Passau are also involved both as organizers and publication contributors.

Figure 36 Workshop on Smart Garments at ISWC 2014 (http://www.simpleskin.org/smartgarments/)
The percentage of co-publication kept growing through the project year, especially cooperation with
external partners. As the basic technologies becoming mature in the late project years, the consortium is
able to not only cooperate more between internal partners, but also attract more external cooperation. The
percentage of single partner publication drops from 75% in 2013 to 44% in 2016, with the external
cooperation of 0% in 2013 to 38% in 2016.
In total we published with 11 external partners 10 peer-reviewed papers, not only with universities and
research institutes within and beyond Europe, but also with industry partners. (6 Germany: Univ. Hannover,
Univ. Saarland, Univ. Freiburg, LMU Munich, FAU Erlangen, Max Planck Institute for Informatics; 1 Finland:
Univ. Tampere; 1 China: University of Science and Technology of China; 1 Japan: Osaka Prefecture University;
2 industry: Yahoo Research, Adidas AG).
. Promoting Researchers
Within Simpleskin project, the promotion of researchers happened on all levels, which created research
groups and researchers, that work with a considerable percentage on smart textile and will continue
contributing to both research and possible industrial applications with smart textile after the end of
SimpleSkin project.
• Prof. Oliver Amft moved from TU/e (Holland) to Uni Passau, Germany and promoted to a full professor,
where he created and holds the chair of Sensor Technology.

Page 35 of 61
 • Prof. Fernando Seoane, at the University of Borås, moved during the project from the School of
Engineering to the Faculty of Care Science, Working Life and Social Welfare, where he has been recently
promoted from Associate rank to full rank professor.
• Prof. Paul Lukowicz, the orignal coordinator, shifted the task to Dr. Jingyuan Cheng (female). The latter
is still young and SimpleSkin is the first EU project she is coordinating. Due to the fact they are working
in the same group at DFKI, Prof. Lukowicz supports her all the time with coordinating tasks. Prof. Georg
Kampis from DFKI, Prof. Oliver Amft from Uni Passau and Prof. Albrecht Schmidt, who are involved in
multiple EU projects, also provided their expertise and support. Dr. Jingyuan Cheng moved later to TU
Braunschweig and promoted to a junior professor, where she created and holds the Wearable
Computing Lab.
• Stefan Schneegass from USTUTT, finished his Ph.D. within the scope of SimpleSkin. The dissertation is
entitled: “Enriching Mobile Interaction with Garment-Based Wearable Computing Devices”, was
defended on July 15, 2016 at the University of Stuttgart. The Thesis was supervised by Albrecht Schmidt
(USTUTT) and members of the examination committee included Paul Lukowicz (DFKI). Its abstract and
the first chapter is attached to WP5.
• 17 students finished their final bachelor/master work within the scope of SimpleSkin.
1) Bo Zhou, Resistive Pressure Force Sensor Matrix for Wearable and Ubiquitous Computing (master
thesis), supervised by DFKI
2) Mathias Sundholm, Activity Recognition Using Floor Based Pressure Sensors (master thesis),
supervised by DFKI
3) Bing Wang, Presentation Evaluation based on Pressure Sensitive Smart Chairs(master thesis),
supervised by DFKI
4) Marco Hirsch, Head Movement Detection and Classification with a Capacitive Textile Neckband
(master thesis), supervised by DFKI
5) Orkhan Amiraslanov, Electroluminescent Based Flexible Screen (master thesis), supervised by
DFKI
6) Linn, Tobias “Input Finger Identification using Wearable Devices” (bachelor thesis), supervised by
USTUTT
7) Röhrle, Lucas “Privacy of Wearable Devices” (bachelor thesis), supervised by USTUTT
8) Ogando, Sophie “Enhancing the Visual Output of Smart Watches using Garment-Based Displays”
(bachelor thesis), supervised by USTUTT
9) Müller, Tamara “Implicit and Explicit Authentication using Electronic Muscle Stimulation”
(bachelor thesis), supervised by USTUTT
10) Bulut, Velihan “Communicating Activity Tracking Data to User” (bachelor thesis), supervised by
USTUTT
11) Alkis, Onur “Die Körperhaltung als implizite Eingabe für Sport- und Rehabilitationsaktivitäten”
(diploma thesis), supervised by USTUTT
12) Voit, Alexandra “Using Touch Sensitive Fabric for Interaction with Smart Devices“ (diploma
thesis), supervised by USTUTT
13) Birmili, Tobias “Entwicklung einer Architektur für das Betriebssystem von intelligenter Kleidung“
(diploma thesis), supervised by USTUTT
14) Mayer, Sven André “Modeling distant pointing for compensating systematic displacements”
(diploma thesis), supervised by USTUTT
15) Severin Bernhart, Integrating Electromyography (EMG) Sensors into Smart Glasses for Chewing
Detection(bachelor thesis), supervised by Uni. Passau
16) Irmandy Wicaksono, Analog Front-end Design for Simultaneous Signal Acquisition of Multi-modal
Textile-sensors (ETHZ)
17) Seminar and practical course dedicated to SimpleSkin (StuPro, USTUTT)

Page 36 of 61
. Lectures to Students
The Univ. partners, namely Uni. Passau, Uni. Stuttgart, Uni. Boras, ETHZ and DFKI, are all holding bachelor
and master courses, demos are shown in the lectures and students are invited to the labs.
Uni. Passau created two bachelor and master student seminars on information extraction from fabric
materials. A new master-level course on wearable and implantable computing is under preparation for the
Mobile and Embedded Systems curriculum and the Intelligent Technical Systems specialization of the
Computer Science curriculum.
USTUTT created seminar and practical course dedicated to SimpleSkin (StuPro). Also a 90 minutes lecture on
Wearable Computing including the presentation of different SimpleSkin technologies to the students was
given.
DFKI created some general resistive sensing platforms and a general data recording and processing library
for resistive sensing. One practical lecture based on these platforms and library is under development and
will be held first at TU Braunschweig in Winter semester 2016/17, then by DFKI at Summer semester.
. Demos to general public
DFKI has demonstrated smart textile developed within SimpleSkin continuously at Cebit, the world's largest
and most international computer expo, including:
• 2014: Smart table cloth (details in D6.1)
• 2015: SmartMat for sport (details in D6.2)
• 2016: Smart leg-band for muscle monitoring and Smart wristband for Human-Computer Interaction as
part of the Wearable Competition Center AI.
These demonstrations have constantly drawn interests from visitors and media.

Figure 37 Smart TableCloth, SmartMat, Wearable Competition Center AI by DFKI at Cebit 2014,15,16
The SimpleSkin project was also presented by DFKI at 14th/19th Wearable Technologies Conference on Feb. 2-
3, 2015 and Jan. 26-27, 2016 in Munich, Germany and to the school girls on 20.05.2015 in Kaiserslautern,
Germany, within the framework of Girl’s Day, funded by the German Federal Ministry for Family Affairs,
Senior Citizens, Women and Youth, the German Federal Ministry of Education and Research, aiming at
opening positive future prospects for girls (http://www.girls-day.de/english).

Page 37 of 61
4.1.5 Project Website and Contact Details

www.simpleskin.eu or www.simpleskin.org

Partners Contact person Contact

Deutsches Forschungszentrum für Jingyuan cheng j.cheng@tu-bs.de
Künstliche Intelligenz GmbH Paul Lukowicz paul.lukowicz@dfki.de
(DFKI) Bo Zhou bo.zhou@dfki.de
University of Stuttgart (USTUTT) Albrecht Schmidt Albrecht.Schmidt@vis.uni-stuttgart.de
Stefan Schneegass Stefan.Schneegass@vis.uni-stuttgart.de
Deutsche Institute für Textil-und Hansjürgen Horter karl.goenner@itv-denkendorf.de
Faserforschung Denkendorf (ITV) Karl Gönner hansjuergen.horter@itv-denkendorf.de
Eidgenoessische Technische Matija Varga Matija.varga@ife.ee.ethz.ch
Hochschule Zürich (ETHZ) Andreas Mehmann Andreas.mehmann@ife.ee.ethz.ch
SEFAR AG Peter Chabrecek peter.chabrecek@sefar.ch
Werner Gaschler werner.gaschler@sefar.ch
Universität Passau Oliver Amft amft@fim.uni-passau.de
Rui Zhang rui.zhang@uni-passau.de
Martin Freund martin.freund@uni-passau.de
HÖGSKOLAN I BORAS Fernando Seoane Fernando.seoane@hb.se

Page 38 of 61