David Hall is a tech innovator and inventor. While his official title is that of a distinguished engineer, his colleagues call him a visionary, an influencer, a tech evangelist, and an instigator. But his journey to tech visionary is far from typical.
From Iowa to innovation
Hall was born in a small town in Iowa and moved nearly every year until high school at which time his single mother wanted to put down some roots and offer stability to Hall and his sister.
He went to college to study psychology but didn’t stay long.
“I wasn’t very good at psychology because I realized it’s based more on opinion than fact, which did not jive with my nature,” said Hall. “I’ve always liked building things and taking them apart, so I failed completely out of school.”
He decided to go into construction and started a career in sheet rocking, building interior walls and ceilings in houses. After stints as an auto mechanic and a truck unloader, he returned to sheet rocking but soon thereafter decided it was too dangerous and physically intensive.
He petitioned the University of Northern Iowa to take him back and began studying physics. That career path was thwarted when the Superconducting Super Collider particle acceleration project in Waxahachie, Texas was canceled; suddenly job prospects for physicists were greatly reduced.
Hall picked up a book on computer programming and fell in love. He aced college the second time around and went on to earn a master’s in artificial intelligence (AI), genetic algorithms, and machine learning at Michigan State University. Married with two young children, he stopped short of pursuing a Ph.D. and landed his first job at IBM in Rochester, New York.
Hall’s career with Western Digital has unfolded over 24 years. He holds more than 55 patents. Specializing in firmware, algorithms, AI, and machine learning, his inventions span the HDD, SSD, and architecture realms. Through his inventions, he has made significant contributions to the company’s continued capacity and performance gains.
“What I learned from those twists in the first half of my life is to seize the moment for a do-over, and there’s almost always another door to open,” he said. “Reach for the stars but also know when it’s time to move on.”
Envisioning invention
It’s this attitude that has propelled Hall’s out-of-the-box thinking that brings ideas to invention.
In his tenure at Western Digital, Hall has always sought ways to push the limits and improve technologies.
“Anticipating change is an inventor’s most valuable skill,” says Hall, who looks to the future when brainstorming ideas. His first step is to extrapolate what will happen if technology continues its current path. Then, at what point will it break and how can that be mitigated?
“Anticipating change is an inventor’s most valuable skill.”
Over the years, Western Digital engineers have increased HDD capacity either by adding more storage disks and heads or packing more density into each disk. But eventually, alternatives such as firmware enhancements and algorithms had to be considered.
One such algorithm was for adjacent track interference (ATI) refresh, which kept a lookout for any data corruption that might occur when tracks are packed closer together.
“Before ATI management, you could write to an adjacent track an infinite number of times. Then as more tracks were squeezed together, that number went from 10,000 to 1,000 to 100 to now sometimes only six,” Hall explained.
As tracks were squeezed closer together, frequent ATI refreshes began to overwhelm the system and impact performance.
To solve this, Hall invented XCOR. XCOR helped ATI by filtering writes so that the damage counter for a track did not need to be incremented on every write. It would check for overlap between writes by subdividing the track into segments. If overlap was not detected, the counter for the track wouldn’t be incremented. This increased the number of random writes that could occur before a track needed to be refreshed.
XCOR fit into the existing DRAM and enabled capacity gains because of its lower refresh thresholds as writes decreased (from 10,000 down to six.)
“For an ‘algorithmnist’ like me, I’m always looking for ways to work around constraints and minimize impact,” said Hall.
For an ‘algorithmnist’ like me, I’m always looking for ways to work around constraints.”
Constraints as opportunities for innovation
Hall’s overarching belief is that constraints should be viewed as an opportunity, not a burden.
“Constraints are an opportunity to have an invention that uniquely adapts to those constraints,” he explained.
One of his first patents was for the Delta Expected Access Time (DEAT) invention. Building upon work done at Almaden Research, which included expected access time and not just access time, Hall created an algorithm that optimized performance by measuring miss rates in expected access time for data writes. As a result, a 300K table was reduced to 1K which was a 99% reduction in size that came with a 35% increase in performance.
One method he uses to tackle constraints is to imagine there are no constraints and remove all ideas that bind a team to its current solution.
“The best way to address constraints is to ignore them for a second,” said Hall. “This is what allows the space for ideas.”
Once you remove those constraints, Hall continues, you can tweak your solution to find a better way to work around it. For example, how can you get 80 to 90% gains for 10 to 20% of the cost?
“The best way to address constraints is to ignore them for a second. This is what allows the space for ideas.”
Rumination nation
Sometimes the best ideas need to percolate over time.
Hall’s first invention was media cache, which he originally called “write twice cache.” While media cache is now an established way to improve write performance by storing data within the media itself, it took almost 15 years for his idea to come to fruition. Sometimes for an invention to take hold it needs to solve the right problem at the right time.
In another example, Hall had been thinking about new ways to use NAND back in 2015, even before Western Digital’s acquisition of SanDisk.
As Hall looked at the cost curves for HDDs and NAND, he could see a shrinking differential between the cost of making HDD media and flash, and the capacities were closing that gap.
“I knew there had to be some point at which that gap gets small enough that you’d like to use some NAND with HDD,” Hall said.
Fast forward to 2021 and the introduction of OptiNAND™, a flash-enhanced drive architecture that improves performance by storing metadata in non-volatile flash instead of on the rotating media. OptiNAND takes ATI refresh one step further by keeping track of every sector on a drive and how many writes have been performed.
Collaboration
The evolution to OptiNAND entailed a collaboration between Western Digital’s HDD and SSD engineering teams who were united by a common goal to increase capacity and performance. Hall recommends talking and listening to peers, mentors, and even critics to bring ideas to fruition.
Hall has had many mentors in his career and has advanced to management himself. Today he manages a team of firmware engineers in the HDD business unit.
“As a manager, I’ve learned the role is much different. I go back to my psychology roots – it’s about what motivates each person,” he said. “I’ve realized management (and mentoring) is about giving your team freedom to innovate. You realize their success is your success.”
He finds satisfaction in watching the next generation carve new paths to innovation. He cites two more inventions that helped one of the company’s hyper scaler customers strike the right balance between performance and power savings.
“I brought a brilliant young engineer to the whiteboard and said, ‘Do you want to be a hero?’” recalls Hall. “I described to him the things that we needed and he, in very short order, implemented them in the drive, had prototype code, and we ended up shipping them. What we got from both performance and power savings really made us industry leaders.”