Mysterious Files PH

Tuesday, March 31, 2026

Improving FDM Filament Drying With a Spot of Vacuum

March 31, 2026 0
Improving FDM Filament Drying With a Spot of Vacuum
Art of 3D printer in the middle of printing a Hackaday Jolly Wrencher logo

Keeping your filament safely away from moisture exposure is one of the most crucial aspects of getting a good 3D print, with equipment like a filament dryer a standard piece of equipment to help drive accumulated moisture out of filament prior to printing or storage. Generally such filament dryers use hot air to accomplish this task over the course of a few hours, but this is not very efficient for a number of reasons. Increasing the vaporization rate of water without significantly more power use should namely be quite straightforward.

The key here is the vapor pressure of a liquid, specifically the point at which it begins to transition between its liquid and gaseous phases, also known as the boiling point. This point is defined by both temperature and atmospheric pressure, with either factor being adjustable. In a pressure cooker this principle is for example used to increase the boiling temperature of water, while for our drying purposes we can instead reduce the pressure in order to lower the boiling point.

Although a lower pressure is naturally more effective, we can investigate the best balance between convenience and effectiveness.

Vapor Pressure Of Water

The main thing that determines whether or not a substance is in a liquid or gaseous state is pressure from the surrounding gas, specifically the surrounding air or equivalent. Although some of the liquid’s molecules will gradually make their way into these surroundings, at e.g. atmospheric pressure at sea level you do not expect to see water instantly boil-off, whereas nitrogen and oxygen are fortunately all in a gaseous state until either very high pressures, very low temperatures or both.

How easy it is for a liquid to transition to a gas depends on its volatility, which itself is related to the strength of its intermolecular interactions. If these are rather weak then a liquid will transition into a gaseous state relatively easily, meaning at lower temperatures and lower pressures. For water this transition point at sea level is at about 100°C, but for people who live a km or more above sea level, this boiling point starts dropping rapidly.

These principles are used in a variety of ways, with many kitchens featuring a pressure cooker: this is a special pressurized pan that increases the boiling point of water by increasing the pressure inside the vessel, thus speeding up cooking times.

Vapor pressure schematic overview. (Source: Wikimedia)
Source: Wikimedia

Increasing pressure of a gas can also turn it back into a liquid, as is the case with for example liquefied petroleum gas (LPG) which is generally stored in pressurized containers. Similarly, in the case of liquefied natural gas (LNG), natural gas is gaseous at atmospheric pressure and room temperature, but is a liquid at -162°C, with some level of pressure above that atmospheric pressure also required. LNG superseded purely pressure-based storage methods in the form of CNG, which requires pressures over 200 bar (>20 MPa).

What we’re trying to do with heating up 3D printer filament and bags of forbidden candy is thus to increase the energy in the system, bringing it closer to the point where the trapped moisture can overcome the vapor pressure of the surrounding air and escape. Logically this means that if we can reduce the surrounding pressure by removing as much of the atmospheric gas as possible, this moisture can escape significantly easier.

Essentially what we need is a pressure cooker, just one that reduces pressure.

Inverted Pressure Cooker

The relation between pressure and temperature as far as the vapor pressure of water is concerned is well-documented. Intuitively at 0 Pa water will boil off practically instantaneously, as there is no vapor pressure from a surrounding atmosphere. The question for our purposes is however just how much we need to reduce the pressure to make a difference, i.e. how deep of a vacuum we need.

Water saturation pressure graphic. Credit: Engineering Toolbox
Credit: Engineering Toolbox

Looking at a relevant graph, such as this one from the Engineering Toolbox site, we can see that the relationship between pressure and temperature is fairly linear below atmospheric pressure at sea level at 100 kPa (1 bar). Rather than trying to hit some arbitrary point on this curve, we should instead look at what off-the-shelf options we have available that may work for us here.

Since there’s no need for us to hit some kind of ultra-high vacuum, it would be plenty to hit something below 1 kPa, which is absolutely achievable with even a consumer-grade roughing pump like a rotary vane pump. This type of pump is commonly used for silicone and resins in hobbyist applications, making it a solid first target. Theoretically these can vacuum dry filament and more at room temperature.

Another option we have are diaphragm pumps, which come in piston- and eccentric variants. These have the advantage of not requiring oil, and do not produce vaporized oil on their output that has to be captured or vented. They do not hit quite the same vacuum levels as rotary vane pumps, but they can be quite easily staged to improve the final vacuum.

Hot Or Not

Even with most of the gases evacuated around the material that we’re trying to extract moisture from, we still have the option to add thermal energy to hurry the water molecules along. If, for example, we can only hit a pressure of around 100 mbar, we would still need to raise the temperature significantly above room temperature to get the intended effect.

Even with the same PTC-type heater as used in off-the-shelf filament dryers, we could still save significant power and time as now the boiling temperature of the trapped water is less than 50°C. Whether or not this is a very significant difference is something which can be ascertained experimentally after we first get a baseline on what difference just changing the environmental pressure makes.

Thus, all that remains is obtaining some data by firing up a gaggle of vacuum pumps and writing down the results.

Running Experiments

A typical cheap single-stage rotary vane pump kit. (Credit: Maya Posch)
A typical cheap single-stage rotary vane pump kit. (Credit: Maya Posch)

The most straightforward experiment involves the use of a budget rotary vane pump and associated vacuum chamber. Here I picked up a Vevor 3.5 CFM single-stage rotary vane pump (model KQ-1K) rated for 150 Watt along with an 11 L vacuum chamber. Unfortunately the first pump that I received was defective and sounded like someone had lost a bag of spanners inside it while running, while only hitting a sad final vacuum of ~400 mbar.

Fortunately the replacement unit seemed to work a lot better and hit -1 bar on the chamber’s vacuum gauge along with a happy burst of nebulized oil from the pump’s air-oil separator. It was finally time to load up the chamber with some wet things.

Samples prior to loading into vacuum chamber. (Credit: Maya Posch)
Samples prior to loading into vacuum chamber. (Credit: Maya Posch)

As testing the moisture content in a spool of filament is tricky at best, I instead opted for two much easier indicators of vacuum drying chops: a bag of color-changing (cobalt(ii) chloride-containing) silica desiccant and juicy pieces of fruit (apple and banana). The latter items being mostly because it’s a fun experiment and dried fruit is tasty, plus it’s another way to judge drying capacity.

After loading in the samples, the chamber had a vacuum pulled, with the pump managing 10-20 mbar. This is approximately one light-year away from the advertised 5 Pa, but then nobody trusts marketing on non-laboratory equipment. Other than there being clear bubbling/boiling of fluids being visible on the apple piece as the vacuum formed there was little to observe.

After letting it rest for approximately 24 hours the chamber was checked and confirmed to have retained its vacuum level. Ignoring physical changes, the samples’ weight were compared to their pre-vacuum exposure. This gave the following results:

  • Apple: originally 53.23 grams, final weight 51.76 grams. Decreased 1.76 grams.
  • Banana: from 41.64 grams to 40.58 grams. Decreased 1.06 grams.
  • Desiccant: from 3.12 grams to 3.78 grams. Gained 0.66 grams.

This shows that the fruit definitely lost some moisture, while the silica desiccant wasn’t saturated yet and kept doing its thing. As for the effect on the fruit, the apple looked fresh and other than a slightly dryer outer layer was still moist and tasty. The piece of banana had however turned gooey and was not very appetizing any more.

As an aside, the Vevor pump also got rather hot after a few minutes, with cloudy oil in the reservoir, so the best way forward here might be to invest in a second-hand twin-stage lab-level pump instead.

Diaphragm Time

Two diaphragm vacuum pump setups with high-tech jam jar vacuum vessel. (Credit: Maya Posch)
Two diaphragm vacuum pump setups with high-tech jam jar vacuum vessel. (Credit: Maya Posch)

With those results in hand, we still got two more vacuum setups: the two types of diaphragm pumps. Both are readily available via any online shopping platform, with the micropumps available for about $5 a pop, as they’re commonly used in e.g. vacuum packaging devices. The larger eccentric pumps are also found everywhere, but come in significantly pricier, even if they can pump a much larger volume per minute.

Weighing the forbidden candy before some vacuum time. (Credit: Maya Posch)
Weighing the forbidden candy before some vacuum time. (Credit: Maya Posch)

Here the micropumps are connected in a four-stage configuration, while the eccentric pumps feature a two-stage configuration. Both use the same vacuum chamber, being a repurposed glass jam container. Not only is jam rather tasty, their glass jars are also designed to maintain a vacuum for extended periods of time as part of the preservation process, making them excellent small vacuum chambers.

We run the same experiment as before, but only with the silica desiccant. This shows a rather similar outcome, just with these pumps not hitting quite the same final vacuum. For the twin-stage eccentric pump setup the final vacuum was about 100 mbar, and the quad-stage micropump system hit 60 mbar.

Much like with the rotary vane pump experiment, there was no clearly visible color change to the desiccant. The weight remained unchanged from an initial 3.26 grams, taking into account the variability of those cheap ‘precision’ scales, even after calibration.

Discussion

These filament dryers would look better with some vacuum. (Credit: Maya Posch)
These filament dryers would look better with some vacuum. (Credit: Maya Posch)

What these experiments make clear is that merely having a low vapor pressure isn’t a silver bullet when you want to remove moisture from silica desiccant or unsuspecting pieces of fruit. It also shows why vacuum packing foodstuffs is a good way to keep them fresh for longer, as leaving a piece of apple lying around on a kitchen counter for a day would result in a far less tasty result.

The application of thermal energy is thus apparently not just a good idea, but might be the best way to make moisture hurry up in evacuating from a sample, especially when water is bound to e.g. a desiccant. For the next stage of this vacuum drying adventure we’ll thus be looking at putting vacuum chambers into some kind of thermal chamber, like a confused mixture of an autoclave and pressure cooker. Here the main question is the selection of the optimal heating solution, which is where again there are many choices.

This should reveal whether the <100 mbar of the cheaper, 12 VDC-powered diaphragm pump setup is enough to make it a competitor to the mains-powered rotary vane pump for this purpose. Beyond this there is also the question in how far existing filament dryers could be retrofitted to support some level of vacuum, as well as potential vacuum storage.

Any thoughts, notes, references and more on this topic are most welcome.


Reverse-Engineering The Holy Stone H120D Drone

March 31, 2026 0
Reverse-Engineering The Holy Stone H120D Drone
A laptop communicating with the drone via an Arduino

There are plenty of drones (and other gadgets) you can buy online that use proprietary control protocols. Of course, reverse-engineering one of these protocols is a hacker community classic. Today, [Zac Turner] shows us how this GPS drone can be autonomously controlled by a simple Arduino program or Python script.

What started as [Zac] sniffing some UDP packets quickly evolved into him decompiling the Android app to figure out what’s going on inside. He talks about how the launch command needs accurate GPS, how there’s several hidden features not used by the Android app, et cetera. And it’s not like it’s just another Linux SoC in there, either. No, there’s a proper Real-Time Operating System (RTOS) running, with a shell and a telnet interface. The list of small curiosities goes on.

After he finished reverse-engineering the protocol, he built some Python scripts, through which you can see the camera feed and control the drone remotely. He also went on to make an Arduino program that can do the latter using an Arduino Nano 33 IoT.


Monday, March 30, 2026

Retro Open Source Camera Straight from the ’90s

March 30, 2026 0
Retro Open Source Camera Straight from the ’90s

In our modern society, we have started to take the humble camera for granted. Perhaps because of this, trendy standalone cameras have started to take off. Unfortunately, most of the time these cameras are expensive and not any better than those in our everyday smartphones. If only there were some open-source solution where you could build and customize your own standalone device? [Yutani] has done just that with the SATURNIX.

Simple microcontrollers and cameras meant for Raspberry Pis are a dime a dozen these days. Because of this, it’s no surprise to hear that the SATURNIX is based on recognizable hardware, a Raspberry Pi Zero 2W and an Arducam 16MP sensor. The Pi Zero powers both the sensors’ capture abilities and the interactive LCD display.

Some sample filtered shots from the SATURNIX

With a simple visual design, the device could certainly fit into the same market we see so many other standalone cameras. Pictures from the camera look great without or with the included filter options if you want a more retro look. While currently there do appear to be some speed improvements needed, the best part of open source is that you yourself can help out!

We always love ambitious open source projects that look to build a true base for others to work on, and this seems like no exception! If you want similarly impressive feats of optical trickery, look no further than using scotch tape as a camera lens!


Recreating One of the First Hackintoshes

March 30, 2026 0

Apple’s Intel era was a boon for many, especially for software developers who were able to bring their software to the platform much more easily than in the PowerPC era. Macs at the time were even able to run Windows fairly easily, which was unheard of. A niche benefit to few was that it made it much easier to build Hackintosh-style computers, which were built from hardware not explicitly sanctioned by Apple but could be tricked into running OSX nonetheless. Although the Hackintosh scene exploded during this era, it actually goes back much farther and [This Does Not Compute] has put together one of the earliest examples going all the way back to the 1980s.

The build began with a Macintosh SE which had the original motherboard swapped out for one with a CPU accelerator card installed. This left the original motherboard free, and rather than accumulate spare parts [This Does Not Compute] decided to use it to investigate the Hackintosh scene of the late 80s. There were a few publications put out at the time that documented how to get this done, so following those as guides he got to work. The only original Apple part needed for this era was a motherboard, which at the time could be found used for a bargain price. The rest of the parts could be made from PC components, which can also be found for lower prices than most Mac hardware. The cases at the time would be literally hacked together as well, but in the end a working Mac would come out of the process at a very reasonable cost.

[This Does Not Compute]’s case isn’t scrounged from 80s parts bins, though. He’s using a special beige filament to print a case with the appropriate color aesthetic for a computer of this era. There are also some modern parts that make this style computer a little easier to use in today’s world like a card that lets the Mac output a VGA signal, an SD card reader, and a much less clunky power supply than the original would have had. He’s using an original floppy disk drive though, so not everything needs to be modernized. But, with these classic Macintosh computers, modernization can go to whatever extreme suits your needs.

Thanks to [Stephen] for the tip!


Medieval Alhambra’s Pulser Pump and Other Aquatic Marvels

March 30, 2026 0
Medieval Alhambra’s Pulser Pump and Other Aquatic Marvels
Reflective pool of the Court of the Myrtles, looking north towards the Comares Tower. (Credit: Tuxyso, Wikimedia)

Recently the Practical Engineering YouTube channel featured a functional recreation of a pump design that is presumed by some to have been used to pump water up to the medieval Alhambra palace and its fortress, located in what is today Spain. This so-called pulser pump design is notable for not featuring any moving parts, but the water pump was just one of many fascinating engineering achievements that made the Alhambra a truly unique place before the ravages of time had their way with it.

Although the engineering works were said to still have been functional in the 18th century, this pumping system and many other elements that existed at the peak of its existence had already vanished by the 19th century for a number of reasons. During this century a Spanish engineering professor, Cáceres, tried to reconstruct the mechanism as best as he could based on the left-over descriptions, but sadly we’ll likely never know for certain that it is what existed there.

Similarly, the speculated time-based fountain in the Court of the Lions and other elements are now forever lost to time, but we have plenty of theories on how all of this worked in a pre-industrial era.

Alhambra

Evening panorama of Alhambra from Mirador de San Nicolás, Granada, Spain. (Credit: Slaunger, Wikimedia)
Evening panorama of Alhambra from Mirador de San Nicolás, Granada, Spain. (Credit: Slaunger, Wikimedia)

A UNESCO World Heritage Site since 1984, the Alhambra saw its first construction in 1238 CE by Muhammad I, the first Nasrid emir. The Nasrid dynasty would last from 1238 to 1491 CE when the Muslim state of al-Andalus fell during the Christian Reconquista.

Even after the end of the Nasrid dynasty would the Alhambra see further construction by Charles V in the 16th century. This made the Alhambra a rather unique amalgamation of Islamic and Renaissance-era architecture and engineering. Sadly by the 18th century the structure had been abandoned for centuries, invaded by squatters, and partially destroyed by the troops of Napoleon in 1812.

Only after these troubled times did an appreciation for such cultural heritage begin to flourish, with European and American tourists alike frequenting the area. One of them – US author Washington Irving – was so inspired by his visit in 1828 that he’d end up writing Tales of the Alhambra, containing many myths, stories, sketches, and essays pertaining to the site. This book in particular was instrumental in making an international audience aware of this site and its legacy.

This renewed attention resulted in the site becoming recognized first as a Spanish Cultural Heritage monument in 1870 and subsequently by UNESCO more than a century later.

Water Features

Most fortresses of the era relied primarily on water cisterns that collected rainwater, as well as access to local rivers in some form, usually requiring human or animal labor to transport the latter. This was also how the Alhambra started in its initial fortress form, called the Alcazaba, meaning ‘citadel’ in Spanish, from Arabic al-qaá¹£abah. The water from this cistern didn’t just supply drinking water, but also for the bathhouse (hammam) and water elements like a pool or fountain for houses in the interior urban area. These houses additionally featured latrines that were flushed using this cistern water.

As the Alhambra expanded, with many palaces and related structures added, its water requirements increased correspondingly. Rather than some small decorative water features for a dozen houses and a communal bath, there were now reflective pools, fountains and a much larger population. This necessitated finding more efficient ways to get more water up the hill on which the Alhambra was constructed.

Aqueduct of the Alhambra as it enters the wall. (Credit: Sharon Mollerus, Wikimedia)
Aqueduct of the Alhambra as it enters the wall. (Credit: Sharon Mollerus, Wikimedia)

In addition to the aforementioned pump, there was also an aqueduct (the Acequia Real) that carried water from the Darro River. At a distance of 6.1 km from the fortress the river is at a sufficiently high elevation to provide water using just gravity. This aqueduct additionally provided water via additional branches to gardens and settlements beyond the Alhambra’s walls.

Many details can be found in this 2019 summary of applied hydraulic techniques at al-Andalus fortresses by Luis José García-Pulido and Sara Peñalver Martín.

As noted in that overview article, the reason for the Alhambra being significantly more advanced than other fortresses in the al-Andalus region was that it was the seat of the Nasrid dynasty, ergo it was only natural that it’d not only get all the palaces and comforts, but also the most advanced technologies for supplying water.

Unfortunately the unique pumping device that was used to supply the Alcazaba with water from the aqueduct was replaced in the 18th century with a more basic syphon system and the original device was removed. Up till that point the previous device had continued to work, despite the new owners of the Alhambra not understanding its operating principles. This left 19th century researchers like Cáceres to essentially fully rely on notes made during the previous century.

That said, there are also hints that the Alcazaba of the Antequera fortress used a similar device to pump water uphill, featuring ceramic pipes and other features that are described in by Sancho de Toledo in 1545. Unfortunately these accounts were all written by people who lacked the engineering know-how of the original Nasrid engineers – or any engineering knowledge at all – and thus had no understanding of the workings of these pumps.

This means that we will unfortunately never know exactly what this device looked like or how it worked, but we can still look at some mechanisms which we are familiar with today that could have been used. The concept of the hydraulic ram or pulser pump would seem to come closest compared to what little we do know.

Self-Powered Pumps

 1) Inlet - drive pipe; 2) Free flow at waste valve; 3) Outlet - delivery pipe; 4) Waste valve; 5) Delivery check valve; 6) Pressure vessel (Source: Wikimedia)
1) Inlet – drive pipe; 2) Free flow at waste valve; 3) Outlet – delivery pipe; 4) Waste valve; 5) Delivery check valve; 6) Pressure vessel (Source: Wikimedia)

Unlike a water pump that uses e.g. an impeller to impart kinetic energy and thus move the liquid, a self-powered pump uses physical phenomena like the water hammer effect or the fact that gas in a liquid will rise in order to effect a pumping effect. The hydraulic ram, for example, uses the water hammer effect and relies only on the kinetic energy of the incoming water.

The basic hydraulic ram functional sequence involves the water current pushing the normally open waste valve close, at which point the water hammer effect from the sudden current cessation forces the delivery valve open and pushing water into the delivery pipe.

This process will reverse again after a short while,  sending a pressure wave upstream and eventually leading to the waste valve reopening. The downstream flow will then resume again, restarting the whole process.

In terms of technological complexity this is a very straightforward design, with the most complex parts being the valves and the pressure vessel that cushions the system against pressure shocks. This is however a design that would have been technologically quite feasible to manufacture and operate.

Basic pulser pump design. (Credit: Belbury, Wikimedia)
Basic pulser pump design. (Credit: Belbury, Wikimedia)

Another, similar type of pump is the gas lift pump. A very small variant of this is commonly used in devices like coffee percolators, with the pulser pump being in effect a very large implementation of the same general principle. Rather than applying heat to the water reservoir in order to create gas (i.e. steam), the pulser pump uses an air compressing effect that’s also used with water-powered trompe air compressors.

As water falls down a pipe it drags air bubbles along with it, which eventually arrive at the bottom where said air is trapped in a cavity while the water flows on to a lower elevation.

The thinner pipe through which water ultimately is pumped is inserted into this air chamber in such a way that it’ll alternately ingest water and air as the level of the latter varies over time. This way pockets of water become trapped between pockets of air, with a resulting pulsing output of water at the end of this pipe.

Whether the original device at the Alhambra or Antequera exactly matches either pump design will likely remain forever a mystery, but neither were beyond the technological means of the time, with the pulser pump arguably even more straightforward due to a lack of need for any valves and pressure vessels.

Time Or Reflective Fountain

Although the Practical Engineering video focuses on this pump design, its author – Grady – was inspired by a Primal Space video that’s basically just history slop content, not citing any proper sources and propagating myths and misinformation as fact. The worst offender is probably the myth that the fountain that is found in the Court of the Lions was time-activated, with the only evidence for it being a clock being that there are twelve lion statues and there are two times twelve hours in a day.

Court of the Lions and its fountain in 2021. (Credit: Sean Adams, Wikimedia)
Court of the Lions and its fountain in 2021. (Credit: Sean Adams, Wikimedia)

When we consider the archaeological evidence that exists so far, as well as the findings during the recent restorations, it seems clear that the marble block with its many holes through which the water entered the bowl was intended to diffuse the flow. Around the bowl we can see a corresponding poem of twelve verses by the vizier and poet Ibn Zamrak.

In verses 3 through 7 it specifically refers to “[..] which runs to that which is still, that we know not which of them is flowing”. This quite strongly suggests that the theme was similar to that of the many reflective pools that were so popular around the Alhambra and elsewhere. The idea of it being a time-controlled mechanism would thus seem to be a purely Western interpretation, barring some hitherto unknown evidence appearing.

Lossy History

Perhaps the most cruel aspect of history is that, much like time itself, it has no concern for those of us who live in the present. Throughout the eons as empires rise and crumble back into dust, wondrous inventions are made and soon again forgotten, leaving behind only echoes of deeds and wonder.

If we’re lucky some of it is recorded in a form as durable as Sumerian clay tablets buried underneath desert sands, but if not then what once was shall never be again. This impermanence is the eternal curse of the past, and also the reason why it’s always so important to make multiple copies of your important data.

Due to the passage of time history is mostly just ruins, pot shards and bones buried in mud and sand. Some will try to spruce things up with one’s imagination resulting in faux romanticism, but this naturally bears little connection to the past. That today the Alhambra has been largely restored is testament to how much more respectful we now approach the past, but the parts that were erased after the demise of the Nasrid dynasty are sadly likely to be lost forever.

Featured image: Reflective pool of the Court of the Myrtles, looking north towards the Comares Tower. (Credit: Tuxyso, Wikimedia)


Tame the Tape: Open-Source Dotterboard for Bulk SMT Parts

March 30, 2026 0
Dotterboard smt counter

One of the great things about building electronics today is how affordable SMT components have become — sometimes just fractions of a cent each. That low price often means ordering far more than you need so you’ll have spares on hand the next time a project calls for them. Keeping track of exactly how many of each part you actually have, though, is rarely easy. To solve that problem, [John] built the Dotterboard, an open-source SMT tape counter.

While working on some of his other projects, [John] found himself managing thousands of tiny SMT parts and decided it was time to automate the counting. The Dotterboard takes inspiration from the BeanCounter — a compact, portable SMT tape counter — but expands the design to handle larger components beyond the 8 mm tapes the BeanCounter targets.

The Dotterboard is mostly 3D-printed and uses just a few common hardware parts such as springs and ball bearings. An OLED displays the current count, which comes from an encoder tracking movement and multiplying by the number of components per hole. At the heart sits an RP2040 Zero that needs nothing more than a single USB-C cable for power, unlike the bulky industrial SMT counters that demand AC outlets and desk space.

Be sure to check out all the details of the build on [John]’s website, and grab the files from his GitHub if you want to make your own. Let us know what are some projects you’ve done to save you the headache of doing the same task by hand for hours on end.


Sunday, March 29, 2026

Training a Transformer with 1970s-era Technology

March 29, 2026 0
Training a Transformer with 1970s-era Technology

Although generative language models have found little widespread, profitable adoption outside of putting artists out of work and giving tech companies an easy scapegoat for cutting staff, their their underlying technology remains a fascinating area of study. Stepping back to the more innocent time of the late 2010s, before the cultural backlash, we could examine these models in their early stages. Or, we could see how even older technology processes these types of machine learning algorithms in order to understand more about their fundamentals. [Damien] has put a 60s-era IBM as well as a PDP-11 to work training a transformer algorithm in order to take a closer look at it.

For such old hardware, the task [Damien] is training his transformer to do is to reverse a list of digits. This is a trivial problem for something like a Python program but much more difficult for a transformer. The model relies solely on self-attention and a residual connection. To fit within the 32KB memory limit of the PDP-11, it employs fixed-point arithmetic and lookup tables to replace computationally expensive functions. Training is optimized with hand-tuned learning rates and stochastic gradient descent, achieving 100% accuracy in 350 steps. In the real world, this means that he was able to get the training time down from hours or days to around five minutes.

Not only does a project like this help understand these tools, but it also goes a long way towards demonstrating that not every task needs a gigawatt datacenter to be useful. In fact, we’ve seen plenty of large language models and other generative AI running on computers no more powerful than an ESP32 or, if you need slightly more computing power, on consumer-grade PCs with or without GPUs.