Tl;dr: this is about an art project i conceived at The Borderland 2023 and set up at The Borderland 2024 for the first time. If you don’t know what The Borderland is, what Burns are or what any of this means, worry not. It’s not that hard to google.
The Idea
At The Borderland 2023, just as many others, I wandered around the forests, inspired by the many lights illuminating them. Two things struck me though: one, that they always relied on some form of physical connection, be it electricity or just the properties of being an LED strip.
This limits the light project. I immediately thought „i want _vast_“. So i came up with the following criteria to build my own project
- The lights should be able to light up on their own. Battery powered.
- The lights should be still able to synchronize themselves and blink in a coordinated manner
- The lights should be bright
- The batteries should last for the whole week, blinking every night
- I wanted a way for the lights to know their spatial position
- The whole project should be extendable with additional hardware.
- I want a one page instruction set for anybody to set the entire thing up.
First steps
At Chaos Communication Camp 2023 i talked to a lot of folks who know things about hardware, radio frequencies and all the other stuff. We debated LoRa (too expensive) vs RF24 (maybe) vs ESP Now (yes), we debated whether to do the spatial position recognition via Wifi or Bluetooth triangulation (too complex) vs GPS (not in a forest, also imprecise), vs Audio based (actually doable?). We debated whether to build our own hardware (quite an effort, but scalable) or rather stick some existing components together (at scale much more of an effort).
So after some more research, there were a few really clear tasks to be done:
- Implement a time synchronization mechanism between ESP NOW devices
- See, if with that time sync we can estimate the distance of a sound source to the device
Both worked, sort of.
First prototype
Learning KiCad was quite a ride. But i have a good friend who helped me through this. I learned about ground pulls and plane fills, about the necessity of a voltage regulator and a charging ic with deep discharge protection. In the end there was a prototype that sort-of worked. It had an ESP32-S2 on board, two high power 3w LEDs, some MOSFETs to drive them, a MEMS microphone and an amplifier.
There were a bunch of faults
- The voltage regulator wasn’t powerful enough and shut down the system upon peaks (especially when sending esp_now messages.
- Turns out, i had to desolder the LEDs and place some Kapton tape underneath, as the middle part actually was mislabeled in the datasheet
- I had a great idea for charging: instead of using USB i wanted to just stick all devices on a model railway. This is kinda cool but highly impractical, so while i left it in for historical reasons i didn’t use it anymore.
- Leaving out the USB port and using the UART pins instead turned out to be way more annoying than anticipated
- Leaving out boot / rst buttons and adding pins instead also turned out to be super annoying
- Somehow we had misread the datasheet and assumed the microphone would have a pre-amp inside. It didn’t and therefore was useless.
- I also noticed that implementing the entire audio-out-thing, getting the amp running (it didn’t, immediately), implementing something cool, thinking about a design for the casings that would include a speaker, would eat up too much time. So i decided to drop it for now and just add some pin headers for GPIOs in the next version
Second Prototype
By now it was about January and I mainly worked on the second prototype, fixing these issues. Adding a pre-amp for the microphone (and switching to an electret mic), rewiring the LEDs, adding some capacitors, adding a USB plug, replacing the switch and also accomodating for a different casing.
I had originally thought about a „push in and twist“ mechanism for the boards, but it became clearer that i wanted to keep the entire thing together. Yes, screws, but also more stability, which the original design just didn’t have. Also it turned out to be quite finnicky and i wanted something that others could use as well.
I also noticed that the original design had the hanging wire „built in“ and figured it would be better to make it removable. Smart choice, it turned out, transport wise.
So, by end of March i had this design ready and prototype 2 in my Mailbox.
I now had made the decision, to put a stricter deadline on myself. Instead of Borderland (last week of July), i would put up my project at Kiezburn, third week of June. Why? Well, first of all to actually test the stuff. Second, because funding. I was running way over my initial 3000 Euro Budget, running close to 5000 and that wasn’t including the microcontrollers, which i was getting gifted.
So i applied for dream grants, and luckily got them. I also received 500 Euros in grants from Mauersegler e.V. for the prototypes. Thank you!
There were only a few mistakes left. I had decided to go with two battery ports. there was a huge mixup in the importing of the materials list that caused about ten resistors to be just wrong. And in my complete overwhelm i was enternally grateful for Bruno to help me out on this, so i shipped my boards to Sweden, he resoldered them and sent them back. Phew.
I also wanted to have some more stability, so i included two more layers in the PCB for proper GND / VDD access, and I finally decided to go „two-sided“, just because of arrangement issues. The ESP32, the USB port, the switch _and_ the microphone all needed to go on the bottom edge which i wanted to keep as slim as possible.
I also decided that for proper sleep mode i wanted a proper time-giver so i added a 32khz crystal. To the wrong Pins, as it turned out later, but that’s a story for chapter four.
With this being done, i decided to go for prototype 3.
Third Prototype
Some of my friends at the Chaos Communciation Camp Badge Project still had a bunch of ESP32 lying around, which i was able to donate. Thank you so much! So Prototype 3 was supposed to also test those. And what a fuckup i had when i found out that i had botched the order.
So in an absolute crazy last minute effort Felix picked up a board, stripped an ESP32 S3 from a dev board and soldered it on to mine. Which i couldn’t have done, lacking tools and skills. Thank you, Felix!
With that working I decided to place the order. Only to find out that JLCPCB (the company manufacturing the PCBs) had run out of the right LEDs. In a night of panic i tried to source the LEDs from other places, then got told by JLC that in fact only the order number had changed. Still, i lost two valuable days. And when i then found out, after placing the order, that my pins for the Crystal were wrong, i pulled the entire order and re-placed it, losing another four days.
Deadlines
Time was ticking. I had placed the order for the batteries mid April, but they were nowhere to be seen. Due to disruption in the shipping world (Houthi Rebels shooting rockets at ships in the red sea) and other screw-ups in communication, i wouldn’t receive the batteries in time. So i luckily found a guy in Hamburg who strips 18650 batteries from old e-bike battery packs and he could help me with at least a few. The final PCBs wouldn’t come either. In fact, the delays and some chinese holidays meant, they would eventually show up on my doorstep on friday of Kiezburn.
I decided to go anyway, and while a lot of things failed, and programming (see different chapter) dragged along, i managed to set up ten lamps, blinking, on friday night. And it was beautiful.
With three weeks to go to The Borderland I now had to assemble everything. Which brings me to…
The Casing
These are pictures of the casing. Essentially the following criterias drove me to the final design:
- Design needed to be rain proof
- The outer shell needed to be printable as fast as possible. Hence the ellipse (faster than round), hence the thickness (or lack thereof)
- The same goes for „fitting as many casings as possible on one print bed“. Round would have meant „only four“, elliptical: six.
- I didn’t want to print anything with supports
- The battery casing needed to be replaceable in itself, in case cables would rip or other things would happen
- The batteries shoudn’t block out any light
- Accessability to the board (usb, switch, reboot/bootloader buttons) mandatory
- Removability of the hook part.
So essentially every casing consisted of five parts. A base, a top, a screw with a hole and two battery holders. Which i inserted connectors into and soldered wires on.
This all works quite well and during multiple rain storms none of them has caused failures. However, they break easily, so i’m considering doing a better outer casing. I also added some screws on the inside to keep the boards in place.
The entire design is done in OpenSCAD, so it’s very modular.
Simulator
My original plan was „well, if i want to code animations, i can hardly do that while sitting in a forest, so i want a simulator“. Some really nice person helped me a little in Godot, but it didn’t really do the trick. Then Tomek (thanks!) built me something in Unity. Which at least produced a really nice demo. But i didn’t end up using it much. I might, though, in the future.
Code
So much code. So many fuckups along the way. This was also a journey of me coding the first real software project in 15 years, since i had chosen to do project management and then becoming a CTO rather than coding after i had completed university. And ESP32 is incredibly badly documented. I also chose the hard way in terms of programming language, using C++ rather than micropython.
My very undocumented, insanely chaotic code is here. https://github.com/hdsjulian/sparkles – i will tidy it up later.
In the future i intend to clean up the state machine, clean up the messaging. Then move everything to FreeRTOS tasks. Make sure timer interrupts and wifi stuff don’t mess with each other (which they won’t, due to FreeRTOS). Possibly add a „hardware check“ routine when starting up the device to make sure, that i’m not hanging up faulty devices. I also need to make sure that if a device fails it is removed from the system. which means notifying every device that this is happening.
ESP NOW has a really weird limitation in that you can only add 20 peers. Which means if you are using a master device and a huge number of clients, you can either use broadcast and never get an ACK or maintain the device list with adding devices to the peer list before send and removing them afterwards. Every message draws power, so it is quite important to somehow household a little.
Calibration
To make use of the microphone my friend Danny and I built this clapping device. It registers when closed, sends a message to all devices saying „hey, i clapped“, then the devices know that when they register a clap on their microphone, it must be a signal and not noise. Thanks to the timer i know when the clap occurred and therefore can tell the distance. I set the point of clapping in a web interface and measure the distance from one point to the other with a laser distance measurement device. It sort-of all works, but i haven’t really tested it „in action“ yet, as when i finally wanted to test it at the event, sound stages were already pumping bass and causing interferences.
The laser distance measurement device is also sort of „not perfect“, given that a) you can’t really see the laser pointer during daylight and b) trees aren’t put up in straight lines, so it’s kinda hard to measure their positions without doing too much calculations. Still, i have ideas for this and it will work.
One big task for the future is to, instead of doing „real time clap detection“ rather play a modulated frequency on a boombox, record a second or two of sound on each devices and then analyse that sound snippet to determine the „sound arrival time“.
Time Sync
To make sure lights can blink at the same time, but also in order to get the calibration via audio right, it was important to know the more or less exact time of the master device. Unfortunately esp now is unreliable with most messages taking about 1.5ms to send but some taking up to 10ms So i did the following:
–
- Using a timer interrupt master sends out a „this is my exact time in microseconds“ message every 500 milliseconds
- Using the esp now „onDataSent“ callback functionality i determine the round trip it took to send a message and have it acknowledged
- This round trip is sent with the next message.
- If the interval between two messages is more or less exactly 500ms (give or take a few hundred microseconds) the message is considered „valid“. This means that there was no unnecessary delay and i can assume that i now know the master’s more or less exact system time in microseconds
- After doing this for a few time i can create an average between the delays as well as the offsets for additional accuracy.
This works pretty well, with about an accuracy of 1ms. good enough!
The Borderland 2024
Everything was sort of right on time. I finished the last casings just in time to pack, i finished most of the code just in time to leave for Sweden. In the end I had to do some overtime in the Barn of the Borderland, just to get things done, but after a small test run, a huge amount of desparation and a literally last minute Bugfix, i was sitting in the Forest with my Friends Binni, Nilz, Joe and Jan, hanging up lights.
By the time we had hung them up, by the time it got dark, i was exhausted. And then… Nothing worked. The lights were just rebooting themselves.
I was desolated. So i was told to let it go. Either it would work or it wouldn’t, i wouldn’t get to hang down 160 lights at night anyway. So i went on to see a DJ set of a friend, check out the Borderland, just ignore the fact that i had spent the last 12 months for mostly nothing.
And then this happened:
And what a beauty it was. Hanging there. Illuminating the Forest.
The sleep mode didn’t work, so it only hung for two nights, but those two nights were quite some magic that is hard to capture in a video. Here’s my best attempt:
I also set it up in a smaller space afterwards, but: no videos of that. Still i learned a lot. Setting up takes about 60-90 seconds per light. Taking the lamps down and storing them, takes about 30 seconds per light. For 2025 i aim to have a system that can be set up by anyone using a one A4 page instruction sheet.
Hamburn & Midi
For Hamburn I added a Midi Piano and honestly this might be the final form. It just works so well, adding BTMidi to my code and mapping it to keys was _very_ simple and There’s a bunch of more ideas. Most importantly, playing the piano and seeing the lights blink sparks immense joy and people absolutely loved it.
Outlook
Oh dear, there’s _so_ much to do. Beat Detection for surrounding music stages. Pitch detection for instruments being played into a microphone. Spatial Animations. Using the PCB and creating a Boombox for Spatial Audio… And we’re just getting started. The Animations can do some rework too. It could look better, it could work with less hassle.
I’m also wondering about a different kind of casing. especially the „opening mechanism“. Being able to take the batteries out without having to pull screws would be amazing. We’ll see what the future brings. Collaborators are always welcome and if you don’t know how to reach me just leave a comment.