We have reviewed the documentation, but we have not found any practical examples that could assist us. There is no code example for a Braille display using HID anywhere on the internet. This complicates matters because without an example, it becomes difficult, especially for those who are not familiar with HID. It would be practical to have an example code. In fact, what we are looking for is to create a library for HID Braille, similar to the libraries available for Arduino and ESP32, that converts it into a keyboard. Users can simply load the keys they want, and the library will generate the reports and all other necessary tasks. It would be ideal to have a HID Braille library that can create Braille displays of different sizes, for example, a 20-cell display that uses specific pins, and the library can create the corresponding report list. If someone could provide an example code with the main buttons connected to some of the ESP32 pins, such as the button to read the next or previous message, forward and backward arrows, and serial console output that displays the Braille content. It would be very helpful to have this example code. This would allow us to connect it to the Braille reader, display the Braille on the console, and navigate the text using buttons. This would be a good starting point to continue working with all the required functions. I have been requesting assistance for over a year and have sought help from many programmers. However, none of them have been able to provide any code. Currently, we have been struggling for almost a week, spending entire nights trying to generate code and understand the explanations, but have not been successful.

I'm afraid not. I have only a small amount of knowledge about the display driver side of things, definitely not about how it works in the display itself (i.e. how a display should behave like a braille HID driver).

"Thank you for responding, but let me tell you that I have tried all the libraries related to HID that I could find, and I have tried every possible method that I could imagine. I have also looked for programmers who could help me, but none of them have been able to provide a solution. They all give me theoretical advice on what to do, but there is a big gap between theory and practice. What I really need is someone who can apply their theory and show me an example code that actually works. The computer recognizes the device as HID, so the problem is not with the computer recognizing it. The problem is with the screen reader recognizing it correctly and sending content to it.

I have managed to write some code that makes the computer recognize the device as HID, but NVDA doesn't recognize anything. Obviously, there are specific parameters that need to be placed in certain locations, and these can only be achieved with an example code that shows how to set those parameters.

On the other hand, the HID examples that I have found online only show how to send key press data from the device to the computer, but there is no example on how to receive data from the computer to the device, which is the case for Braille where I need to receive the content of the Braille cells."

"In the case of the projects you mentioned, such as the Android HID project and USBD-Human-Interface, they provide HID support only for USB devices and do not work with Bluetooth devices. In my case, I am facing a problem with the ESP32, which does not have native USB support for HID Braille. I can only work with it through Bluetooth.

However, as I mentioned before, the problem is not that the computer does not recognize the device as HID. The problem is that NVDA, or more specifically HID Braille, has a specific structure to define the cells and many other things, which is not the same as the structures for a keyboard or a mouse. These libraries that convert HID do not have a structure for HID Braille. Therefore, even if I manually change the descriptor and add the HID and Braille descriptor, the computer recognizes it but the screen reader does not, because there are many things that are not properly structured.

That's why it is necessary to create a specific library for HID Braille or add HID Braille support to the existing library."

"If that table specifies how to create the HID structure, it doesn't explain how to work with it, how to send content, how to receive content, or where to load the Braille cells. There is no information on how to update or send data to the screen reader, what needs to be sent, or received. Simply specifying the structure allows the device to be recognized as HID on the computer, but the screen reader will not recognize it as HID without a proper report including information on the number and type of Braille cells (whether 8 or 6 points), and other details that are needed for the screen reader to process the information correctly. None of this information is specified in the table."

"This code using the HID structure recognizes the device as HID on the computer, but the screen reader does not recognize anything. Obviously, the HID structure is only a part of what is needed and other information is missing from the document."
mi código

"Well, to be honest, I have read all the files related to HID personally, but I couldn't understand them because it seems to explain how the reader reads the content, but there seems to be a huge difference between how it is read and how it is sent. Apparently, the structure that one sends for HID is sent in one way for the computer to process it and delivered in another way, so apparently, the structure that is sent is not the same structure that the screen reader receives. In theory, it could be said that if I understand how the screen reader reads it, I will understand how to send it, but you can try it in practice and see that it's not as simple as the theory says. I don't know much about programming, but I have searched for many people and we have all been struggling with the same problem and none of us have been able to do anything."

"The HID protocol is a communication protocol that works on both USB and Bluetooth. In fact, that's why there are Bluetooth keyboards and mice as well as gaming devices that work via Bluetooth, and the computer recognizes them perfectly as HID devices. So, HID devices are not only for USB. In fact, the code I posted in a previous response, if you put it on an SP32, the computer will recognize it as a Bluetooth HID device."

"I think the real problem in developing HID Braille display drivers is the lack of understanding of how the HID protocol works. There isn't a deep knowledge of the protocol, which is why even very skilled programmers haven't been able to do anything with it. It would require someone who specializes in the HID protocol, and it seems that there are few people who master it or it's difficult to find someone who does. It would be interesting if some of the programmers showed more interest in this, especially since the future expects all Braille displays to exclusively use HID. The hope for the future is that even operating systems will automatically recognize a Braille display and connect it without needing a screen reader. When the HID protocol was created, it was done with that intention, just like when you connect a keyboard, whether it's Bluetooth or USB, both work with the HID protocol and are perfectly recognized by the device you connect it to. However, the population that works to develop Braille devices is very small, and in the end, the ones who end up developing direct drivers are the manufacturers. The manufacturer, as such, is not interested in having their code seen by another manufacturer. So we are in a situation where there is no example of the code currently available in open source, and my post is precisely calling for help to see if someone can do something that gives us a starting point, a code that gives us a point to build a good driver on the Braille display side."

That's what I've been understanding over time as I've been looking into it, that depending on the perspective you approach it from, the focus is different. I see that the available documentation is for developing a driver that understands a braille display, but the perspective from the braille display (firmware) is different. From what you're saying in this message, I understand that not all braille displays send information in the same way. They use the HID protocol but have their own commands. I'm not sure if I understood that part correctly, but if so, then unification still hasn't been achieved because it would still be necessary to make the screen reader able to understand each manufacturer's commands. My proposal is to create a public unified command set that allows for specific commands to be developed for a braille display. This document should be made public so that anyone who wants to develop something with those commands can do so, and to allow the screen reader to understand those commands. This way, we would be developing a standard within the HID protocol that anyone looking for information can find and use. But as long as everyone works separately and creates their own commands, we will never achieve the true goal of the HID braille protocol, which is plug and play functionality. While it may be recognized by the operating system, being recognized by the operating system alone is not enough for it to be functional. It would be good to be able to develop a communication standard with the basic buttons required for a braille display and explained documentation on how to implement it with example code that serves as the foundation for anyone to use those commands to implement it in their development.

Something very similar is happening with regular drivers, each manufacturer has its own protocol and commands. So, for each device, an individual driver must be created. In fact, that is the big problem. Hopefully, there would be documentation with a universal driver that anyone could use code or a regular driver with all the documentation so that anyone could replicate the same driver and use it. For example, an open source driver with a recognition standard called "open" for Bluetooth, and any device that has the word "open" at the beginning of the Bluetooth name will be recognized as a device that uses that driver. I think that by implementing it in a screen reader and implementing the first braille display that uses that driver, it would open the door for the driver to be used in the rest of the screen readers. Then, you could truly have the possibility of working with an open source screen reader software and hardware.

@Adriani90

Eso es lo que he ido entendiendo a lo largo del tiempo mientras lo investigaba, que dependiendo de la perspectiva desde la que lo abordes, el enfoque es diferente. Veo que la documentación disponible es para desarrollar un controlador que comprenda una pantalla braille, pero la perspectiva desde la pantalla braille (firmware) es diferente. Por lo que dices en este mensaje, entiendo que no todas las pantallas braille envían información de la misma manera. Usan el protocolo HID pero tienen sus propios comandos. No estoy seguro de haber entendido bien esa parte, pero si es así, todavía no se ha logrado la unificación porque aún sería necesario que el lector de pantalla fuera capaz de comprender los comandos de cada fabricante. Mi propuesta es crear un conjunto de comandos unificados públicos que permitan desarrollar comandos específicos para una pantalla braille. Este documento debe hacerse público para que cualquier persona que quiera desarrollar algo con esos comandos pueda hacerlo, y para permitir que el lector de pantalla entienda esos comandos. De esta forma, estaríamos desarrollando un estándar dentro del protocolo HID que cualquiera que busque información pueda encontrar y utilizar. Pero mientras todos trabajen por separado y creen sus propios comandos, nunca lograremos el verdadero objetivo del protocolo HID braille, que es la funcionalidad plug and play. Si bien puede ser reconocido por el sistema operativo, ser reconocido por el sistema operativo por sí solo no es suficiente para que sea funcional.

Algo muy similar está pasando con los drivers regulares, cada fabricante tiene su propio protocolo y comandos. Entonces, para cada dispositivo, se debe crear un controlador individual. De hecho, ese es el gran problema. Con suerte, habría documentación con un controlador universal que cualquiera podría usar código o un controlador normal con toda la documentación para que cualquiera pudiera replicar el mismo controlador y usarlo. Por ejemplo, un controlador de código abierto con un estándar de reconocimiento llamado "abierto" para Bluetooth y cualquier dispositivo que tenga la palabra "abierto" al comienzo del nombre de Bluetooth se reconocerá como un dispositivo que usa ese controlador. Creo que al implementarlo en un lector de pantalla e implementar la primera pantalla braille que usa ese controlador, abriría la puerta para que el controlador se use en el resto de lectores de pantalla. Entonces,

Hello, I am also trying to make the braille display but I cannot establish communication.
Could you share your code?

Hi everyone, after a year of working on this code with the help of AI, I believe I've got a good grasp on how it works. However, it's not functioning as expected. I'm hoping someone can help me identify the issue.

#include <Arduino.h>

#include "BLEDevice.h"

#include "BLEHIDDevice.h"

#include "HIDTypes.h"

#include "HIDBrailleDisplay.h"



#define DEVICE_NAME "Standard HID Braille Display"



bool isBleConnected = false;

bool hasSentIdentification = false;



BLEHIDDevice* hid;

BLECharacteristic* input;

BLECharacteristic* output;



uint8_t REPORT_MAP[] = {

    0x05, 0x41,        // Usage Page (Braille)

    0x09, 0x01,        // Usage (Braille Display)

    0xA1, 0x01,        // Collection (Application)

    0x1A, 0x01, 0x02,  // Usage Minimum (Braille Keyboard Dot 1)

    0x2A, 0x08, 0x02,  // Usage Maximum (Braille Keyboard Dot 8)

    0x75, 0x01,        // Report Size (1 bit)

    0x95, 0x08,        // Report Count (8 bits for 8 dots)

    0x15, 0x00,        // Logical Minimum (0)

    0x25, 0x01,        // Logical Maximum (1)

    0x81, 0x02,        // Input (Data, Var, Abs)

    0x09, 0x02,        // Usage (Braille Row)

    0xA1, 0x02,        // Collection (Logical)

    0x09, 0x03,        // Usage (8 Dot Braille Cell)

    0x15, 0x00,        // Logical Minimum (0)

    0x26, 0xFF, 0x00,  // Logical Maximum (255)

    0x75, 0x08,        // Report Size (8 bits)

    0x95, 0x1E,        // Report Count (30 cells)

    0x91, 0x02,        // Output (Data, Var, Abs)

    0xC0,              // End Collection (Logical)

    0xC0,              // End Collection

    0xC0               // End Collection (Application)

};



struct BrailleIdentificationReport {

    uint8_t reportID = 1;

    uint8_t page = 0x41;  // Usage Page (Braille)

    uint8_t usage = 0x01; // Braille Display

    uint8_t numCells = 30; // Número de celdas Braille

    uint8_t cellType = 0x03; // 8 Dot Braille Cell

};



class OutputCallbacks : public BLECharacteristicCallbacks {

    void onWrite(BLECharacteristic* characteristic) {

        uint8_t* data = characteristic->getData();

        int length = characteristic->getLength();



        if (length > 0) {

            Serial.print("Received Braille Cells Report: ");

            for (int i = 0; i < length; i++) {

                Serial.print(data[i], HEX);

                Serial.print(" ");

            }

            Serial.println();

        }

    }

};



class BleKeyboardCallbacks : public BLEServerCallbacks {

    void onConnect(BLEServer* server) {

        isBleConnected = true;

        BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));

        cccDesc->setNotifications(true);

        Serial.println("Client has connected");

    }



    void onDisconnect(BLEServer* server) {

        isBleConnected = false;

        hasSentIdentification = false; 

        BLE2902* cccDesc = (BLE2902*)input->getDescriptorByUUID(BLEUUID((uint16_t)0x2902));

        cccDesc->setNotifications(false);

        Serial.println("Client has disconnected");

    }

};



void sendBrailleIdentification() {

    if (!hasSentIdentification) {

        BrailleIdentificationReport idReport;

        input->setValue((uint8_t*)&idReport, sizeof(idReport));

        input->notify();

        hasSentIdentification = true;

        Serial.println("Braille Display identification sent.");

    }

}



void bluetoothTask(void*) {

    BLEDevice::init(DEVICE_NAME);

    BLEServer* server = BLEDevice::createServer();

    server->setCallbacks(new BleKeyboardCallbacks());



    hid = new BLEHIDDevice(server);

    input = hid->inputReport(1);

    output = hid->outputReport(1);

    output->setCallbacks(new OutputCallbacks());



    hid->manufacturer()->setValue("hidBrailleStandard");

    hid->pnp(0x02, 0xe502, 0xa111, 0x0210);

    hid->hidInfo(0x00, 0x2);



    BLESecurity* security = new BLESecurity();

    security->setAuthenticationMode(ESP_LE_AUTH_BOND);



    hid->reportMap(REPORT_MAP, sizeof(REPORT_MAP));

    hid->startServices();

    hid->setBatteryLevel(100);



    BLEAdvertising* advertising = server->getAdvertising();

    advertising->setAppearance(0x03C9);

    advertising->addServiceUUID(hid->hidService()->getUUID());

    advertising->addServiceUUID(hid->deviceInfo()->getUUID());

    advertising->addServiceUUID(hid->batteryService()->getUUID());

    advertising->start();



    Serial.println("BLE ready");



    while (true) {

        if (isBleConnected) {

            sendBrailleIdentification();

        }

        delay(1000); 

    }

}



void setup() {

    Serial.begin(19200);

    xTaskCreate(bluetoothTask, "bluetooth", 20000, NULL, 5, NULL);

}



void loop() {

    // Nada en el loop

}

HIDBrailleDisplay.h

#ifndef HID_BRAILLE_DISPLAY_H
#define HID_BRAILLE_DISPLAY_H

#define REPORT_ID_BRAILLE_DISPLAY 1

// Define la estructura BRAILLE_KEYMAP
typedef struct {
    uint16_t usage;
    uint8_t modifier;
} BRAILLE_KEYMAP;

enum BRAILLE_DISPLAY_USAGE {
    BRAILLE_DISPLAY = 0x01,         // Braille Display
    BRAILLE_ROW = 0x02,             // Braille Row
    BRAILLE_CELL_8_DOT = 0x03,      // 8 Dot Braille Cell
    BRAILLE_CELL_6_DOT = 0x04,      // 6 Dot Braille Cell
    NUMBER_OF_BRAILLE_CELLS = 0x05, // Number of Braille Cells
    SCREEN_READER_CONTROL = 0x06,   // Screen Reader Control
    SCREEN_READER_IDENTIFIER = 0x07,// Screen Reader Identifier

    // Router Usages
    ROUTER_SET_1 = 0xFA,
    ROUTER_SET_2 = 0xFB,
    ROUTER_SET_3 = 0xFC,
    ROUTER_KEY = 0x100,

    // Braille Keyboard Dots
    BRAILLE_KEYBOARD_DOT_1 = 0x201,
    BRAILLE_KEYBOARD_DOT_2 = 0x202,
    BRAILLE_KEYBOARD_DOT_3 = 0x203,
    BRAILLE_KEYBOARD_DOT_4 = 0x204,
    BRAILLE_KEYBOARD_DOT_5 = 0x205,
    BRAILLE_KEYBOARD_DOT_6 = 0x206,
    BRAILLE_KEYBOARD_DOT_7 = 0x207,
    BRAILLE_KEYBOARD_DOT_8 = 0x208,
    BRAILLE_KEYBOARD_SPACE = 0x209,
    BRAILLE_KEYBOARD_LEFT_SPACE = 0x20A,
    BRAILLE_KEYBOARD_RIGHT_SPACE = 0x20B,

    // Joystick Controls
    BRAILLE_JOYSTICK_CENTER = 0x210,
    BRAILLE_JOYSTICK_UP = 0x211,
    BRAILLE_JOYSTICK_DOWN = 0x212,
    BRAILLE_JOYSTICK_LEFT = 0x213,
    BRAILLE_JOYSTICK_RIGHT = 0x214,

    // D-Pad Controls
    BRAILLE_D_PAD_CENTER = 0x215,
    BRAILLE_D_PAD_UP = 0x216,
    BRAILLE_D_PAD_DOWN = 0x217,
    BRAILLE_D_PAD_LEFT = 0x218,
    BRAILLE_D_PAD_RIGHT = 0x219,

    // Pan Controls
    BRAILLE_PAN_LEFT = 0x21A,
    BRAILLE_PAN_RIGHT = 0x21B,

    // Rocker Controls
    BRAILLE_ROCKER_UP = 0x21C,
    BRAILLE_ROCKER_DOWN = 0x21D,
    BRAILLE_ROCKER_PRESS = 0x21E,
};

#define BRAILLE_KEYMAP_SIZE (28)

const BRAILLE_KEYMAP braille_keymap[BRAILLE_KEYMAP_SIZE] = {
    {BRAILLE_KEYBOARD_DOT_1, 0},
    {BRAILLE_KEYBOARD_DOT_2, 0},
    {BRAILLE_KEYBOARD_DOT_3, 0},
    {BRAILLE_KEYBOARD_DOT_4, 0},
    {BRAILLE_KEYBOARD_DOT_5, 0},
    {BRAILLE_KEYBOARD_DOT_6, 0},
    {BRAILLE_KEYBOARD_DOT_7, 0},
    {BRAILLE_KEYBOARD_DOT_8, 0},
    {BRAILLE_KEYBOARD_SPACE, 0},
    {BRAILLE_KEYBOARD_LEFT_SPACE, 0},
    {BRAILLE_KEYBOARD_RIGHT_SPACE, 0},
    {BRAILLE_JOYSTICK_CENTER, 0},
    {BRAILLE_JOYSTICK_UP, 0},
    {BRAILLE_JOYSTICK_DOWN, 0},
    {BRAILLE_JOYSTICK_LEFT, 0},
    {BRAILLE_JOYSTICK_RIGHT, 0},
    {BRAILLE_D_PAD_CENTER, 0},
    {BRAILLE_D_PAD_UP, 0},
    {BRAILLE_D_PAD_DOWN, 0},
    {BRAILLE_D_PAD_LEFT, 0},
    {BRAILLE_D_PAD_RIGHT, 0},
    {BRAILLE_PAN_LEFT, 0},
    {BRAILLE_PAN_RIGHT, 0},
    {BRAILLE_ROCKER_UP, 0},
    {BRAILLE_ROCKER_DOWN, 0},
    {BRAILLE_ROCKER_PRESS, 0},
};

#endif // HID_BRAILLE_DISPLAY_H

Thank you @discapacidad5 for sharing your code.

I can run it and get it recognised by VoiceOver but nothing else.

Are you able to receive Braille Cells Reports?

Hello, I am also trying to make the braille display but I cannot establish communication.
Could you share your code?