Video summary
Filamina Gray presents an innovative approach to photography by utilizing the railway network and ferries as moving flatbed scanners, a technique born from her desire to move beyond practical digital imaging. As a fiction writer and ham radio operator who finds standard photography too mundane, she sought a method that combined motion with stillness to create unique images. The core concept involves capturing thousands of vertical lines per second while the camera moves along a track or waterway; by stitching these rapid frames together in post-processing, stationary objects like buildings and bridges are reconstructed into coherent photographs. This process transforms complex transit systems into massive conveyor belts for light, allowing her to capture scenes that would otherwise be impossible with conventional cameras due to motion blur.
The technical execution of this project required overcoming significant hardware limitations and developing custom software solutions. Gray initially experimented with consumer smartphones but found their frame rates insufficient for capturing enough lines without distortion. She eventually acquired a specialized industrial line-scan camera capable of reading nearly 19,000 lines per second at under $100 from eBay, which she mounted in a custom 3D-printed case equipped with accelerometers and GPS sensors to measure movement speed. Despite challenges such as poor radio signals on Boston trains blocking GPS data and the need for manual exposure adjustments due to limited camera preview capabilities, her team successfully integrated these components into a functional rig that could withstand the vibrations of train travel while collecting vast amounts of image data in real-time.
Post-processing proved to be the most arduous phase of the project, earning Gray the nickname "post-processing hell" as she grappled with issues like parallax distortion and color fringing inherent to line-scan sensors. Because the camera captures lines faster than its speed measurement devices can update, her software had to intelligently select specific frames from thousands per second while compensating for acceleration data that was often relative rather than absolute. She also encountered unique optical challenges when switching to a color sensor, which introduced infrared sensitivity causing unnatural green skies and wavy artifacts; these were resolved by adding an IR cut filter to the lens. Through iterative coding efforts involving friends like Maddie, she developed efficient algorithms named "Grindstone" that could process gigabytes of raw data in seconds, allowing for precise manual adjustments to focus and color balance to achieve crisp, realistic results.
Ultimately, this experiment demonstrates how unconventional tools can unlock new creative possibilities by treating public transit infrastructure as a scanning mechanism rather than just a mode of transport. While the setup was cumbersome with its tangle of wires and tape, leading to occasional security encounters abroad where tripods were banned, the resulting images offer detailed perspectives on industrial landscapes and cityscapes that defy normal photographic conventions. Gray's journey highlights the intersection of engineering constraints and artistic vision, proving that even a messy collection of sensors and code can produce stunning visual narratives when applied with persistence and ingenuity to explore the world through an entirely different lens.
Read the full video transcript
[applause]
Yeah.
Hello everyone. So, got a question for
you. Who here likes public transit?
>> Okay. Who here likes weird photography?
>> [cheering]
>> Well, you've come to the right place.
I took this picture in Oakland out in
California using the motion of a ferry
area moving through the container port.
And this is just a small part of a much
wider picture because I could only fit
so much on the slide and the rest is on
my website and it's very detailed. So
now let's I can into it. So now I've got
you intrigued. I should probably
introduce myself. I'm Filamina Gray. I'm
a witch from Boston in the US who's
constantly distracted by a thousand
random ideas and
write fiction. I do ham radio. Oh, and I
take a lot of photos. And while I do
many things, graphic design isn't one of
the ones I'm good at. So, please pardon
my slides. [laughter]
Okay. So now explain how I took that
picture. So,
oh, the camera is constantly capturing a
single vertical line like these ones in
grayscale,
but it's they're a lot thinner. And as
the camera moves, in that case, because
it was on a moving ship, what exactly it
sees is changing. And if I capture the
lines quickly enough and stitch them
together, I can produce a complete
looking image. It's a bit more
complicated than that, but that's
[laughter] why if I'm giving a talk and
getting the results looking good was a
bit tricky but yeah that's the main idea
you might wonder why am I doing this why
am I causing problems for myself well
allergic to doing anything normally
especially photography because it's just
not interesting enough for me you know
normal digital photography is just too
practical and sometimes even normal film
photography is too practical and that's
why I sometimes shoot large format like
you see here.
This whole idea came about because I'd
been vaguely looking into digital
scanning cameras for a while and I
suddenly had the thought, what if the
camera moved and the subject stayed
still? There's been a lot of work on
stationary scanning cameras like this
back for a large format film camera on
the the right side of the slide and that
came out in the '9s. But there hasn't
Yeah, there hasn't been much on having
the camera move camera itself move
because that would be a bit silly and
impractical, right?
So once I had the idea, I had to know if
it would even work at all or if there
was a good reason why I hadn't seen
anything about it. So to do that, I sat
my phone on my office chair and scanned
my sofa by taking a video as it pushed
it along. I wrote some really terrible
code that would grab the leftmost column
of each each frame and slap them
together into an image. And that's what
you see here. It's something distinctly
an image, but it doesn't look that good.
I messed around some more with the
post-processing of it and ended up
doubling every line. So, show up twice,
which looks a bit less squished, but
it's still a mess. We've got these
squish sections and these really wide
sections. And that's because I wasn't
moving at a particularly consistent
speed, and I also wasn't measuring the
speed. And this was my first glimpse
into what I call post-processing hell,
which I'll be spending a lot more time
in as it [laughter] goes on.
Yeah. I then went out and tried it on a
bigger distance. I pointed out out of a
train on the MBTA subway line nearest my
apartment. And that time I tried to
measure speed, but it didn't work very
well. I had my old phone taped to the
seat of the train to use its
accelerometer and I was holding my
current phone to the window to take the
video and it's just it's a bit squished.
There's just not enough lines. There's
just not enough image information there.
We we need more.
And I found a way to get more. This bad
boy is a Basler RUL 204819GM.
And it's called that because it can read
its 1x 2048 pixel image sensor just shy
of 19,000 times per second. It isn't
even capable of doing fewer than 100
lines per second. It's designed to be
pointed at fastoving conveyor belts to
do machine vision stuff to them. But
what what is the train network but a
very complicated conveyor belt?
[laughter]
You you might think it'd be expensive
and yeah, brand new, it would somewhere
in the range of call for price, but I
found it on eBay for significantly less
than that. I think it was less than $100
shipped to me.
And with only a bit of swearing at the
vendor's toolkit, I was able to get an
image out of it. I think this is pretty
good for just moving it freehand through
the air without measuring the speed at
all. I think this might just work.
Oh, wait. [laughter]
Yeah. Then, in order to take it on a
train without having to have three
hands, it needed a case for it. So, I
made some attempts at mechanical design
and asked my friend Brooke to 3D print
it. It came out pretty utilitarian, but
hey, it works. Put a heat set insert in
it to mount to a tripod and left plenty
of space to attach sensors.
And let's take a look at those sensors.
And I probably should have attached them
some way other than blue painters tape,
but it works. Yeah. So, going clockwise,
we've got the the accelerometer. So,
with a bit of maths, we can gauge the
speed the camera and thus the train are
moving at. There's a GPS, but it didn't
end up working how I'd like because the
trains in Boston are a bit too good at
blocking out radio signals. and those
are connected over I square C into a
microcontroller to shunt the data off to
my laptop. The lens on the front is a
Vivitar 28 mil F/2.8 I already had for a
more normal camera. Its field of view is
good enough for most of the shots I'm
trying with it. And this whole thing is
powered off a USBC battery bank and
there's also an Ethernet cable running
to my laptop. So, it's a bit of a cable
spaghetti monster when in action.
>> [snorts]
>> So with the sensors attached, I can give
those a try. These are both these
pictures are both the same capture, but
the top one is just the raw image and
the bottom one is taking accelerometer
movement into account. So as you can see
with the accelerometer, everything looks
a lot less stretched and closer to
normal. I'll explain a bit more of how
I'm taking that into account when I talk
about post-processing hell.
And yeah, so I took it on a train and
started off on the orange line again and
went back and forth a couple times. But
as you can see, the results aren't very
good. It previewing what was coming out
of the camera was a pain. So I kind of
had to guess on the exposure, and I
definitely guessed wrong. And the
post-processing code that I wrote didn't
work very well. and everything is very
oddly stretched and compressed.
But hey, it's an image. Maybe I can get
something good out of this.
Went out again on a day with nicer
weather and had some better luck with
the exposure, but I think I messed up
the focus a bit. Took this one at
Roxbury Crossing Station near the south
end of the MBTA orange line. And and you
can tell that because the text is
legible on the station signs. [laughter]
The ca the catinary wiring you can kind
of see in the background of the second
bit is the Amtrak northeast corridor and
that's why I went in that particular
direction.
And then I'm particularly happy with how
this one came out. I took it while
crossing the Longfellow Bridge from
Boston to Cambridge. And this one's up
on the website so you can see the whole
thing if you'd like. I will admit I
redid this with my new post-processor,
but that mostly just made it it that
just made things easier. It didn't
[snorts]
didn't change the underlying image.
And then this was the software I was
using to preview
at that point. And it comes from the
camera vendor. That small horizontal
black section was all I could see of the
image at any one time. And it's rotated
90° off from how I'd like it. And it's
honestly better than some vendor
software I've encountered. But dialing
in the exposure, stopping it, and
starting my own code code back up all
before the train started moving again
was a right pain.
I got annoyed enough about it that I I
built a gooey about it using deery. Did
most of it in a single sleepless night
in Toronto where it felt like rotating
the image was the single hardest problem
in computer science. But despite that,
it seemed to work. [laughter]
[gasps]
[snorts]
Then, as I went out and took more
pictures, ran into another problem. The
camera and the witch using it both look
incredibly suspicious.
So, this mess of wires going off to my
laptop and into my backpack, and it's
all held together with tape. Sometimes a
lot of tape like this this time. Because
that trip I forgot the tripod mount
plate and [laughter]
had to hold it together with tape, which
definitely didn't help with the goal of
not looking sketching.
So So [laughter]
yeah, so far I've only been seen, not
said or sorted. Knock on knock on wood.
But on my trip to Montreal, I was
stopped by security in the station and
was asked offlay whether I was recording
or taking a picture and was informed
that tripods were not allowed. And
rather than try and answer that
philosophical question, I just said des
a bunch and [laughter] put everything
away.
And here's the photo I was is taking
when that happened. The rest is again on
the website. that particular stretch of
the REM line leading toward Garenthrol.
Some very interesting industrial views.
So with some attempts at pictures taken,
it's time to try and make them actually
look good. This proved to be a pain. And
that's why it's post-processing hell,
not post-processing heaven.
And because the camera is capturing so
quickly, I have more lines than I could
possibly need. But the question is,
which ones do I pick? If I pick too few,
it just looks squished like the photo on
the left of an auto region Quebec. And
if I jump between sections of lines too
quickly, it looks artificial and wrong.
Like you can see if you look at the
waterline on this album cover edit of
[laughter]
the San Francisco one. And then if I
take too many lines, everything looks
stretched and unintelligible.
So to decide that, I ended up using the
speed measured with an accelerometer.
But this comes with a bunch of problems
of its own. Firstly, the accelerometer
isn't actually measuring the speed. It's
measuring the acceleration or the rate
of change of the speed. I can take the
integral of that, but that's relative to
a starting speed.
I can usually assume that the starting
speed is at a station and is thus zero,
but I can't be certain of that. And if
it isn't zero, I have no good way of
knowing what it is and just have to
guess values until I find one that
smells right.
Secondly, as this diagram attempts to
show, oh, the accelerometer I'm using is
only measuring so quickly,
the camera is grabbing lines maybe four
times faster than it. So, every few
lines have to share a value for speed.
It also isn't reading very consistently
because my microcontroller code isn't as
fast as it could be. So, that can cause
some like irregularities in the final
image. And then also how many
acceleration measurements there are
aren't also changes how accurate the
integral is
is and can impact how accurate the speed
is.
You might remember that at back at the
beginning I mentioned putting a GPS
receiver on the camera and while I did
do that it wasn't very useful. It didn't
get a signal on most of the trains I
tried it on. And when it did manage to
get one, it could only read 10 times a
second, which covers like 400 entire
lines.
And if it worked a bit more
consistently, it could be useful for
correcting for integration error. But
that is a problem for future me.
And then even if my speed measurement is
perfect, I still have the problem of
parallax.
things closer to the camera appear to be
moving faster than things further away.
And these three images are each the same
number of pixels out of the out of the
same capture, but a factor of 10 times
slower or each time. And the street
light and sign that are the main thing
in the first image are just a small
detail off to off to the side
at the next power of 10. And then at the
next power of 10, the buildings across
the river come into view because of how
much further or because of how much
further away they are than everything
else. And then the camera the camera and
the software have no way of knowing what
you want to focus on. So I have to pick
that manually for each segment of image.
And it's rather annoying, but I can't
really automate that.
The program that implements all this
post-processing is called Grindstone
because it grinds the multi- gigabyte
raw files down into smaller usable
captures. The first version I wrote was
unbelievably slow and took a couple
hours to process a couple minute long
capture and half the time it wouldn't
even save properly anyway because it had
made it too big for a JPEG.
So, I nerdsite my friend Maddie into
writing a new version that runs a lot
faster, or to use her words, turned it
into a magical girl. And then, since it
isn't accidentally copying the entire
image thousands of times, it usually
finishes up in a few seconds. And then,
since it's so fast, I can try out
various values for the focus and really
dial that in to get very crisp results.
And yeah, while searching eBay one
night, I saw a really good deal on a
color line camera from the same
manufacturer and went for it. It's
slower because it can only capture 9,000
lines per second instead of 18. You that
the monochrome one can, but I think
that's reasonable for getting red,
green, and blue.
And it was pretty much perfect timing,
too, because spring was turning into
summer where I live, and Maddie and I
had had gotten Grindstone into a largely
working state.
And [snorts] I thought it wouldn't be
too difficult, but [laughter]
turns out that three times as many lines
means three times as many problems. And
on the left, you can see these weird
wavy lines under the flag. And on the
right, all those leaves are way brighter
than they should be.
The weird wavy lines and color fringes
turn out to be inherent to how this
camera sensor works. The red, green, and
blue are each separate vertical lines
and thus can't see exactly the same
thing at the same time. And and just
because that's not how normal optics
work.
So you get these fringes when just one
line sees something and it's
particularly noticeable if it's
particular the subject is particularly
bright or dark and then how big they are
big the fringes are depends on how fast
the subject is moving. So it's the same
problem from earlier with parallax. You
can't collect correct for it on the
entire image. And right now I do just
correct for it manually which is a pain.
And then the weirdness with the colors
especially of trees and leaves are
because the color camera can see into
the infrared range. The edge of human
vision is somewhere around 750 nmters.
But the camera is still sensitive if a
fair way is beyond that. And the green
and blue channels are also sensitive
there. So I can't just turn down the red
gain and hope for the best. I have to
stop infrared light from hitting the
sensor somehow. The monochrome camera
can also see into IR, but since it's not
as sensitive there, it doesn't look as
weird.
And solved this with an IR cut filter on
the lens, which doesn't let light longer
than 700 nmters through. And in the top
one, you can see that the colors look
pretty normal and the sky looks blue,
unlike this middle one without the
filter filter where the sky is the same
light green as the trees.
I also tried a 720 nmter filter which
only passes infrared and I'm definitely
going to experiment with that some more
at some point but haven't done so yet.
Yeah, here's one I took without a filter
on the Rockport line north of Boston.
The angle the sun was at made the IR
effects really pronounced. Like if you
aren't looking at the RGB fringing in
the background, you might almost forget
it's a color image.
and it's on the website to take a closer
look at if you'd like.
Then took this one with the IR cut
filter on a ferry heading south from
Boston. And the colors look pretty
normal, but everything is a bit too far
away because I brought the wrong lens.
I think it still looks pretty good, but
there's definitely a lot more to
experiment with.
I'd like to thank my accompllices
Meadow, Brooke, Ari, Maddie, Moano, and
Cat, and for riding trains and fairies
with me and helping out a lot with both
code and mechanical design. If it wasn't
for them, none of this would have come
out anywhere near as well as it did.
And thank you for being here. [laughter]
[applause]