Building better HMIs

With increasing demand to create human-machine interfaces (HMIs) that invite engagement and help operators to quickly achieve the results they want, more sophisticated technology is needed. Graphical TFT-LCDs have become accessible and cost-effective for a wide range of embedded electronics. Typical screen sizes for small, smart devices range from about 1” to 5”, though 7” and larger are now becoming more commonplace. Pixel counts can range from 128x128 through 320x240 (QVGA), 640x480 (VGA) to 800x480 (WVGA) or even 1080x720 for a full HD display.

The fact remains, however, that designing GUIs and integrating TFT-LCDs as part of embedded systems are complex tasks. Creating a GUI demands significant engineering effort and expertise, and controlling the display can require extensive compute power, high-speed frame storage, and interface bandwidth.

Whereas a display like an alphanumeric or segment LCD can be controlled using relatively simple commands, a TFT-LCD needs information for each pixel to render the required image. As the number of pixels that must be addressed rises, with larger display sizes and higher pixel densities, the greater the quantity of data involved. Increasing pixel colour depth - say from 8-bit to 16-bit or 24-bit - also pushes up demands on compute performance, frame-buffer size, and interface bandwidth. At a colour depth of 24 bits per pixel, a 640x480 VGA display requires about 1MB of RAM to accommodate the frame buffer. Moreover, if a high frame rate is needed - for example to render moving images (like animated graphics) smoothly  – then the demands placed on system resources are further magnified.

To ensure an optimal solution, designers must consider the constraints on system power, cost, and footprint, as well as high-level concerns - including time to market and the engineering resources available.

If the embedded host system is a low-cost or low-power microcontroller, a dedicated TFT-controller peripheral is usually not provided. The CPU may not be able to compute and communicate RGB data for each pixel at the required frame rate, while also running the main application, and there may not be adequate on-chip RAM available for the frame buffer. One option, in this case, is to connect the display via an external bridge IC, which is controlled from the system host over a low-bandwidth I2C or SPI connection, and integrates all the necessary frame memory, processing and graphics control elements needed to generate RGB pixel data, as well as synchronisation and backlight control signals. This is the approach pioneered by the FT800 series of embedded video engine (EVE) graphic controller ICs, which was originally developed by FTDI but is now marketed via sister company Bridgetek. As well as driving the display, each EVE chip also integrates additional audio and touch-control circuitry (Figure 1).

Figure 1: Using an EVE chip to control a TFT-LCD.

In order to tackle the hardware and software integration challenges of constructing HMIs, and allow even the most inexperienced of hobbyist to benefit from EVE technology, Bridgetek has created the Cle050 turnkey module. The is a touch-enabled smart display shield that features a 5.0” TFT-LCD, and plugs directly into an Arduino or Mikrobus CPU board. It has additional camera, audio and SD card interfaces and comes with speaker and image sensing accessories options. Bridgetek has overcome any HMI coding challenges that might manifest themselves by linking the Cle050 with the MikroElektronika Visual TFT graphical design environment.

Visual TFT enables designers to execute HMIs layouts by dragging objects like buttons or sliders from the palette into the design area and configuring aspects such as the colour, size, or position. The only coding a designer would thus need to do is to implement any animations that may be required. Visual TFT supports several well-known microcontroller families, including STMicroelectronics STM32, Atmel ATMega, and the Microchip PIC family.

Simple to sophisticated

4D Systems has created three series of ready-to-use TFT-LCD modules that not only simplify integration of the graphical display, but also provide valuable extra functionality. The gen4-IOD ‘Internet of Displays’ family, for example, features built-in Wi-Fi connectivity to help designers quickly create connected Internet of Things (IoT) devices with the added advantage of integrated touch. The modules feature the Espressif ESP8266 Wi-Fi system-on-chip (SoC), which contains a 32-bit microcontroller and 16-bit RISC processor core, with 512Kbyte of Flash memory that allows the module to run a simple user application or act as a slave to a host processor. The module is designed to allow an application board to sit flush on the rear surface, resulting in a slim profile.

The gen4-IOD modules are available with a choice of 2.4”, 2.8” or 3.2” 320x240 QVGA display formats. For a larger choice of displays, up to 7.0" and 800x480 WVGA resolution, 4D’s gen4-ULCD series is based on the Diablo16 graphics processor and is available with an integrated capacitive or resistive touch controller. A more streamlined touch-free option is also available. The Diablo16 has powerful graphics, text, image and animation capabilities, and can operate as a standalone graphics processor, or act as a graphics co-processor to a host controller.

For projects based on a high-performing host system, such as a microprocessor-based one, a simpler display subsystem can be considered. 4D Systems’ LCD CAPEs connect directly to the Beagle Bone Black CPU board, which is powered by an ARM Cortex-A8 microprocessor. With an SGX graphics engine and 512MB of DDR3 SDRAM integrated, developers can take advantage of plentiful resources to build sophisticated, smooth and fluid GUIs. The CAPEs are designed to work with graphical software stacks contained in the Debian Linux distribution.

To help developers design a good-looking GUI to work with any of its broad range of display modules, 4D Systems also supplies the Workshop4 integrated development environment (IDE). Workshop4 provides a selection of environments that allow designers to use the powerful 4DGL graphical language, or alternatively to work in an entirely graphical environment. With the optional features of Workshop4 Pro, a hybrid approach combining graphical and 4DGL elements is possible, and users can also take advantage of the Widget Editor to quickly create sophisticated multi-layer widgets with features such as numbers and movement.

Building your own graphics controller

An alternative approach to this is to use Lattice’s HMI-on-chip (HoC) solution to build a complete subsystem for handling TFT display driving, graphics rendering, touchscreen handling and system interfacing in an FPGA. Lattice has developed this package with technical partner Mikroprojekt, and has created a comprehensive architecture that integrates the Mikroprojekt IQ-Engine graphical software stack to simplify integration of the user interface. Figure 2 illustrates the interactions between the Lattice FPGA and Mikroprojekt software.

Figure 2: The HoC solution contains all the tools needed to build a graphics controller on a Lattice FPGA.

This approach can support intricate graphics effects (like alpha blending) and simplifies deployment by creating a single HMI project file, while at the same time giving freedom from the complexities of working with an operating system. Developers also have flexibility to choose from a wide variety of displays, memories and system interfaces.

HMI design is fully graphical, with no coding required, leveraging the Mikroproject IQ Editor interface-designer application. Support for multiple graphics levels and transparencies, and complex menu structures, in addition to standard elements such as buttons, sliders and bar-graphs help create sophisticated HMIs.

The Lattice ECP3 HMI development kit simplifies access to this software-free approach to HMI design. The kit is based on the standard Lattice ECP3 Versa FPGA kit, and comes with an evaluation version of HoC Editor. It includes a 4.3” TFT touchscreen display mezzanine module that has an SD card slot built-in to aid HMI evaluation and deployment.