We’re excited to host a guest blog from our valued partner, Tuxera. Tuxera is a leading service provider for the automotive industry and provides data storage software that makes automotive systems faster, more reliable and more durable.
Connected cars are now roving data centers. Out of the box, the flash memory hardware storing all this data offers high inputs/outputs per second (IOPS). But software also acts on top of that hardware, transferring data into and out of the storage. To get the maximum potential from the hardware also requires high-performance storage software.
At Tuxera, in addition to embedded storage development, we continually test how file systems affect storage performance. There are many factors that play a role in performance, such as file size, device partitioning, or the file system implementations themselves. But one factor that has often been dismissed or downplayed until very recently is fragmentation.
Contrary to popular belief, fragmentation matters
Fragmentation happens when a file system lays out files in non-contiguous parts, or fragments. (1) It generally occurs because there’s no space left to write new file content next to older file content. Some recent groundbreaking studies into flash memory performance have shown that fragmentation is a bottleneck for or cause for degrading performance.(2,3,4) It’s also a factor we’ve currently been testing in our performance benchmarking lab.
Ext4 is one of the most commonly used file systems in the automotive industry. It’s a good option for low-data or single-stream automotive use cases. Plus, as it’s open-source software it’s free. But when it comes to applications that require handling a lot of data and multiple data streams, things get trickier.
Our customers and partners come to us because they suspect the file system could be a root cause of problems such as frame loss, latency issues, and other performance concerns. We’ve observed from our benchmarking that over the short term, ext4 performs quite well despite fragmenting files. But over the long term, the degree of fragmentation gets worse as more data is written to the storage, and performance consequently drops.
Impacts of fragmentation in automotive systems
What’s more, automotive applications rarely have a single-threaded data stream. Most consist of at least two, if not four or more threads. For example, an automotive dash cam typically has one or two video streams (plus audio where applicable), as well as GPS data and other system-related files (logs, for example) all being written at the same time. Such an application would also “clean up” old files when a predefined free-space threshold is reached. Under this type of workload, our tests show that ext4 becomes heavily fragmented and its performance degrades over time.
Time-lapse showing fragmentation speeds
You can see this in the time-lapse below. Here we show an approximately three-hour test run of ext4 against our proprietary flash-friendly file system implementation, Tuxera Flash File System. Each file system wrote data from a live camera stream, plus four additional streams to simulate sensor data (such as one might find in an ADAS application) to a removable memory card.
The box with cells is a visualization of the data as it’s written to the flash memory blocks. Each color represents a unique data stream. As shown, the data streams handled by ext4 become rapidly fragmented into short segments spread across the memory. At the same time, the red box shows the average write speed and latency. As ext4 becomes more fragmented and the storage fills, speed decreases and latency increases—the latter with some very extreme maximums. Tuxera Flash File System, on the other hand, writes the files in longer, contiguous sequences, and maintains higher speeds with consistently low latency.
Approximately three-hour time lapse showing fragmentation, speed, and latency of ext4 versus Tuxera Flash File System.
*Actual performance may vary based on the hardware, software, and testing protocols used.
What are the potential impacts of fragmentation on user experience?
In applications with intensive reading, writing, and rewriting of data, (such as cameras for autonomous driving) fragmentation may cause anything from small errors to critical system failure. If the storage is full and heavily fragmented, there will definitely be read/write issues. This can’t happen when safety and lives are concerned.
In the case of infotainment, performance loss due to fragmentation boils down to a user experience and customer satisfaction concern. If the in-vehicle infotainment (IVI) storage is heavily fragmented, this means longer wait times for music and navigation apps to launch and be available. A minor inconvenience, for sure, but no one enjoys laggy interfaces. User experience is no small issue when it comes to differentiation for auto makers in the coming years, so even small inconveniences such as app response time cannot be easily dismissed.
Early failure of the storage is another potential consequence of heavy fragmentation. With the ext4 file system, for example, the only way to guarantee a satisfactory user experience is to periodically defragment the file system. But, as Hahn et al. (2017) point out, weekly defragmentation using the conventional ext4 defragmentation utility could reduce the lifetime by up to 10%. (4)
Ensuring user satisfaction and safety in automotive
Various methods to prevent or alleviate fragmentation are needed to guarantee both user satisfaction and safety, as well as extend flash lifetime. Fragmentation is a first-order performance issue (4) —though to what degree in automotive storage? It’s a topic we’re still exploring. What we do know is that the file system interfacing with the hardware will fragment data, which works against all the advantages of and advances in flash technology.
The trick is to reduce fragmentation in the first place, which relies heavily on the file system design itself. A file system needs to be “smart”—carefully engineered to lay out files as logically as possible, with the lowest amount of fragmentation. While preventing fragmentation altogether may be impossible, reducing fragmentation should be an important characteristic of a file system, especially for automotive applications.
Western Digital and Tuxera will be at Embedded World Nuremberg this week. Visit Western Digital at booth 3A-429 and Tuxera at booth 4-547.
References
1. “File system fragmentation”
2. Conway et al. “File Systems Fated for Senescence? Nonsense, Says Science!”15th USENIX Conference on File and Storage Technologies (2017).
3. Ji et al. “An Empirical Study of File-System Fragmentation in Mobile Storage Systems.” 8th {USENIX} Workshop on Hot Topics in Storage and File Systems (HotStorage 16) (2016).
4. Conway et al. “File Systems Fated for Senescence? Nonsense, Says Science!”
Joel Catala is Tuxera’s Director, Product Management of Embedded Solutions. He coordinates embedded product development and plans the technical and business strategy of all embedded file systems. You can find Joel at events and associations serving as an evangelist for Tuxera’s products and services. Joel has a B. Sc. in Computer Systems Networking and Telecommunications from the University of Montpellier II (IUT de Beziers).