Due to the open nature of our driver system, it’s relatively straightforward to write a new driver file for almost any imaginable device in Nebula. However, driver development can be a pain, so I though I’d share some debugging tricks.
For all commands, we will navigate to the command line using windows Command prompt or powershell. Here I show command prompt syntax. Nebula is commonly installed under program files. So in the windows start bar type “cmd” and press enter. This should open the command prompt. The commands below navigate to the install location.
cd
c:>cd "Program Files"
c:Program Files>cd Nebula
c:Program FilesNebula>
First, is the use of logging to the command line. When Nebula is launched from the command line, a series of optional switches can be used to report logged output. By opening this alongside the app, it’s very fast to figure out what or where something is going wrong. To see log output, use –printlog.
c:Program FilesNebula>Nebula.exe --printlog
This launches the app, but shows a live log output on the console at the same time.
Get a full list of Supported Drivers
This command will return a complete list of drivers found in the Nebula supported list. To print this list, use:
c:Program FilesNebula>Nebula.exe --listdrivers
Here is live example of that command and the expected result. A list of drivers is shown, press enter to close the listing.
Testing a specific Driver File
Next, let’s load up one of these drivers in the test utility. Now, this is where things get cool! The test utility can load a host device driver, inspect it’s shown devices, attempt to connect them, and even test every sub-device!
So, let’s assume we have a new camera driver we want to work on. The camera should do things like snap an image, show various properties etc. In my example I use a real UVC (web) camera, and inspect the driver. The driver actually pulls the real camera info and even snaps a test image. The goal of this function is twofold, first to validate a given driver, and second to validate the device itself.
For this command we will use –testdriver *drivername* and optionally –port *portname*, for example, COM5
Here we can watch the expected output in real time. Note a new window is spawned to show us the results for these tests.
Testing Host and Sub Devices
Let’s say we have a controller, for example, an ASI or Marzhauser box, which has a Z motor attached. In Nebula, the controller can be thought of as the Host (the resident controlling brain), and the device can be considered an appendage attached to the host. So all drivers must have a Host, but a Host may only have one device. In this example, we use a demo Focus device. First the host is loaded and analyzed, and then, you ca select to test any device provided by that host. I choose option # 1 (focus) and the test runs, even showing my driver has a small error I missed!
Obviously, if you have a driver which isn’t behaving, please contact us – yet I hope these options provide a first-line of interface for power users, so simple things can be corrected and analyzed quickly!
While this is an old subject in terms of technology, I find that every few years I end up describing it to a new group. Surprisingly to me, every time people seem to appreciate the information, and I think it’s just not covered often in broader imaging trainings. So today I’m writing about a favorite subject of mine – TTL control. To do this I’ll be referencing some fun technology – namely:
For illustration of the device control, I’ve captured a few real example frames from my oscilloscope, but also added some simulated oscilloscope examples that I hope will better depict the work behind the scenes for these devices.
I think it’s most important to frame this in a question,
When my software controls devices, how fast do they respond?
Assume we click a shutter button in any software. This can be for a laser, an LED, even an old mechanical shutter. In every case a bunch of things must happen for the end device to actually change it’s state. For that state (let’s say, “OFF” to change to “ON”,
the software must accept the command
and issue control signals to the device connection.
That connection talks to the device controller (normally, a microcontroller)
The microcontroller reads the command and does what it’s told
The state changes.
Now, along the way, each device has a communication bus, and each bus has a traffic control system to insure multiple communications don’t conflict. As a result, the timing from the start of the button click to the end is variable – sometimes, up to 20 milliseconds!
Now, because the software is typically programmed to “confirm” each thing it does, it doesn’t just shout orders into the dark. Instead, the software waits for a confirmation, or acknowledgment, of the command being received, and oftentimes it will confirm the new state is active.
Consider this means a new set of jobs which must be completed.
The device controller changes state
The device controller then replies, or reports state
The communication connection on the PC receives the reply
The software reads the reply, and confirms it’s correct
The software continues to the next job.
And so again, we can see a stack-up of not only delay, but a variable amount of delay.
If we visualize this in a graphic example like the one below, we can see the software code is sent to an LED microcontroller, the command is processed, and the led illuminates. However, we can also see there is a lot of room for improvement!
LED Control via Software (PC → LED Controller → GPIO)
A host PC issues serial commands. The LED Controller parses each command and drives a GPIO pin. The oscilloscope captures the resulting digital waveform on PA5.
$ serial /dev/ttyUSB0 115200
> led_on()
ack: OK
> led_off()
ack: OK _
HOST PC
UART · TX
LED Controller
UART → parser
GPIO_WritePin()
LED Controller
GPIO · PA5
LED + 330Ω
115200 8N1
3.3 V digital
OSCILLOSCOPE · CH1 — GPIO PA5 RUN 1V / div · 500 ms / div
(function(){
var root = document.currentScript && document.currentScript.parentNode;
if(!root) return;
var canvas = root.querySelector(‘canvas.abx-scope’);
if(!canvas) return;
var ctx = canvas.getContext(‘2d’);
function resize(){
var rect = canvas.getBoundingClientRect();
var dpr = window.devicePixelRatio || 1;
canvas.width = Math.max(400, Math.floor(rect.width * dpr));
canvas.height = Math.max(150, Math.floor(rect.height * dpr));
}
resize();
window.addEventListener(‘resize’, resize, {passive:true});
Another way to look at this control is what happens with a typical complex instrument. Below is a video where I control 3 devices. The PCO Camera, a Zaber filter turret, and the Tetrem. Notice how long it takes just to get 3 time points for 2 channels. The exposure is short, that’s not the issue – the issue here is of course, the devices need to move, but also each device must receive + process + move + report positions, and this takes a long time.
TTL Control -> Old + Fast = Awesome
So what is TTL? Basically, nothing more than an on/off switch! Just like a light switch allows an electrical signal to feed a light, TTL uses a voltage to send a signal to another device. You can find a thorough history on this communication method here if interested.
What we care about is that we have an on/off control system. Because it’s nothing more than a voltage, no crazy translations need to happen for things to immediately react. If a TTL control line is on, or “High”, then the device should be doing whatever ON is. Many devices have this input, but few people even know about it! For example, most Focus controllers, XY Stages, Peristaltic Pumps, Perfusion systems, and many other devices like lasers, shutters, and actuators accept a TTL input for control.
TTL Logic typically comes in 2 flavors. Older systems will use 5V for “on” and 0V for “off”. The device being controlled might use thresholds or a midpoint. Some devices, assuming you give them at least 2.5V, will go active. Others want at least 4V or 4.5V before they enable. Newer devices often use 3.3v control. This is the same concept as 5V, just with lower voltages. One word of caution here – before connecting things up, it’s good to confirm the logic of your devices. Generally, the amount of power these use is quite low, and signal inputs won’t break things – but in the case of an exposed microcontroller pin, you can definitely cook a 3.3v input with a 5V line!
TTL Blanking – A partial Solution
So what can be done about this overhead? First, we can consider that the only time we really need the illumination on, is when the camera is truly collecting data. One way to do this is to use the I/O control cables found on most scientific cameras. In my case I routed the expose signal from the edge camera into the Tetrem on a single channel, and then, told the tetrem in my software to use “TTL Control”. With these changes, the LED source now waits for a TTL input to go high, before it enables it’s emission.
So now, let’s assume that when the camera is exposing, we can enable the LED line. The software can act as a logical “AND” if needed, but the line won’t enable unless there is real collection of light by the camera. You can see an example of this below.
A single event fires two paths simultaneously: the PC issues a serial command (software path), while a TTL line drives the controller directly (hardware path). The oscilloscope shows the resulting latency gap.
OSCILLOSCOPE · 3-CH · ALIGNED TIMEBASE RUN 1V / div · 10 ms / div
(function(){
var root = document.currentScript && document.currentScript.parentNode;
if(!root) return;
var canvas = root.querySelector(‘canvas.abx-scope’);
if(!canvas) return;
var ctx = canvas.getContext(‘2d’);
function resize(){
var rect = canvas.getBoundingClientRect();
var dpr = window.devicePixelRatio || 1;
canvas.width = Math.max(400, Math.floor(rect.width * dpr));
canvas.height = Math.max(180, Math.floor(rect.height * dpr));
}
resize();
window.addEventListener(‘resize’, resize, {passive:true});
// —- Cycle = 4s. Normalized 0-1 timing must match the SVG keyTimes. —-
// Trigger HIGH: 0.150 → 0.550
// TTL output: 0.155 → 0.555 (essentially immediate)
// SW output: 0.250 → 0.650 (delayed)
function pulse(t, on, off){
t = ((t % 1) + 1) % 1;
// 1ms-ish rise/fall on screen for clean square corners
var R = 0.004;
if(t < on) return 0;
if(t < on + R) return (t – on) / R;
if(t < off) return 1;
if(t < off + R) return 1 – (t – off) / R;
return 0;
}
function trigAt(t){ return pulse(t, 0.150, 0.550); }
function ttlAt (t){ return pulse(t, 0.155, 0.555); }
function swAt (t){ return pulse(t, 0.250, 0.650); }
var CYCLE_MS = 4000;
var WINDOW_MS = 4000; // one full cycle visible
var start = performance.now();
// Pixel positions for each channel inside the scope canvas
function laneRect(canvasH, idx){
// 3 lanes evenly split, with padding
var pad = 18 * (window.devicePixelRatio||1);
var h = (canvasH – pad*2) / 3;
var top = pad + h*idx;
return { top: top, bot: top + h – 4, mid: top + h/2 };
}
function draw(now){
var w = canvas.width, h = canvas.height;
var DPR = window.devicePixelRatio || 1;
ctx.fillStyle = '#0F0F10';
ctx.fillRect(0,0,w,h);
// grid
ctx.strokeStyle = '#2a2a2a';
ctx.lineWidth = 1;
ctx.beginPath();
for(var i=1;i<10;i++){
var x = Math.round(i*w/10) + 0.5;
ctx.moveTo(x,0); ctx.lineTo(x,h);
}
for(var j=1;j<9;j++){
var y = Math.round(j*h/9) + 0.5;
ctx.moveTo(0,y); ctx.lineTo(w,y);
}
ctx.stroke();
// Lane dividers
ctx.strokeStyle = '#2a2a2a';
ctx.beginPath();
for(var k=1;k<3;k++){
var ly = Math.round(k*h/3) + 0.5;
ctx.moveTo(0,ly); ctx.lineTo(w,ly);
}
ctx.stroke();
var elapsed = now – start;
var samples = Math.min(w, 1400);
var fns = [trigAt, ttlAt, swAt];
var clrs = ['#E6C458','#7BDBD1','#B994FF'];
var names = ['CH1 TRIGGER','CH2 TTL → GPIO PA6','CH3 UART → GPIO PA5'];
For the example, the yellow line is the camera output, or trigger output, the teal line is the LED B brightness result, and the purple line is the software-only reaction. In this example, we can see the delay (latency) of the response, yet in most software, the purple line will illuminate before the camera exposure. What this means for research is your cells are being exposed to excitation energy, and you aren’t even collecting that light onto a sensor! So, we are in effect just cooking cells for fun. NOT GOOD!
Oscilloscope reading TTL from PCO camera
Using the Triggerscope for State Control of Devices
This is great for a single LED, but how can we control more than 1 channel? For this, we would use a logic controller like the Triggerscope Mini! With the mini, the camera expose line is routed to it’s TRIG input, and up to 4 output channels are connected to it’s TTL control pins.
Triggerscope Mini with Tetrem Controller for intensity
With this done, we can assign an active line using the triggerscope. This provides fast easy state switching, and can even be done over multiple devices. Doing this will mean, for this example, we won’t be moving our filter cube device. Instead, we would use a multi-band dichroic. However we can switch the LED excitation, and that’s how we will gain some speed here.
In the example above, we see a high speed switch. This is simply by telling the Triggerscope Mini to change channels, instead of the LED. But – there is still more to do to improve our speed!
Streaming for the Win
The above example uses the triggerscope via USB, and while we only have 1 device, we are still using a “set + capture + set + capture” type of sequence. What if we just told the camera, “run as FAST as you can.” Then, we used the output signal from the camera to count frames, and we told the triggerscope, “every other frame, switch from LED 1 to LED 2, then back to 1”. THIS is the power of “Streaming”, or also called “Sequencing”. With this capability, we pre-load stuff to do into the triggerscope. Then, we arm the triggerscope. With that done, our triggerscope is sitting, locked into a state, simply waiting to receive camera frame TTL inputs. Below is a graphical depiction of this technique.
Now, we tell the camera, “Give us 2 channels * 10 frames = 20 pictures, and do so as fast as possible”. In effect, “give us video of 20 frames”.
In this method, there is zero control overhead. The camera free-runs, the triggerscope switches, and the LED reacts. Below is a video example of this, note also there is a small in-view live feed from a web camera (for all these video samples), and you can see that, indeed, streaming is crazy fast! For the below example, we’ve tied the camera “transfer” line into the triggerscope. This way, the triggerscope recieves a brief pulse before capture occurs. We enable on the “low” side of the signal, and illuminate. You can see for the CMOS style readout on the left of the example, once readout is done, the channel switches.
Trigger Switch — Camera-Driven LED Sequencing via Triggerscope
Each time the camera completes a frame, it emits a TTL pulse. The Triggerscope (Advanced Research Consulting) uses that pulse to advance its state machine and re-route its outputs — alternating TTL1 and TTL2 to the LED Controller so LED1 and LED2 illuminate consecutive frames.
Camera (sCMOS)
SENSOR · 2048×2048
FRAME
N
N+1
· 30 Hz
TRIG OUT · frame complete
TTL pulse
Triggerscope
Advanced Research Consulting
CAM IN
SEQUENCE STATE
STATE 1
→ TTL1 active
STATE 2
→ TTL2 active
ROUTE: CAM IN → {TTL1, TTL2} alternating
TTL1
TTL2
TTL OUT 1
TTL OUT 2
Hardware sequencer · sub-µs latency
LED Controller
MODE · TTL routing
IN1
IN2
CH1 → LED1
GPIO PA5 mirror
CH2 → LED2
GPIO PA6 mirror
LED 1
LED 2
Flow: Camera frame-complete → Triggerscope CAM IN → (state++) → TTL OUT {1|2} → LED Controller IN → LED {1|2}
Use case: hardware-sequenced multi-channel fluorescence imaging — no software in the loop, every frame is locked to the camera.
OSCILLOSCOPE · 3-CH · CAMERA-SYNCHRONIZED RUN 1V / div · ~16 ms / div
(function(){
var root = document.currentScript && document.currentScript.parentNode;
if(!root) return;
var canvas = root.querySelector(‘canvas.abx-scope’);
if(!canvas) return;
var ctx = canvas.getContext(‘2d’);
function resize(){
var rect = canvas.getBoundingClientRect();
var dpr = window.devicePixelRatio || 1;
canvas.width = Math.max(400, Math.floor(rect.width * dpr));
canvas.height = Math.max(180, Math.floor(rect.height * dpr));
}
resize();
window.addEventListener(‘resize’, resize, {passive:true});
// —- Cycle = 4s, matches the SVG keyTimes —-
// Camera frame-complete pulses at t = 0.45-0.47 and 0.95-0.97
// TTL1 HIGH: 0.00-0.47 and 0.97-1.00 (wraps)
// TTL2 HIGH: 0.47-0.97
function camAt(t){
t = ((t % 1) + 1) % 1;
var R = 0.003;
function pulse(t, on, off){
if(t < on) return 0;
if(t < on + R) return (t-on)/R;
if(t < off) return 1;
if(t < off + R) return 1 – (t-off)/R;
return 0;
}
return Math.max(pulse(t, 0.450, 0.467), pulse(t, 0.950, 0.967));
}
function ttl1At(t){
t = ((t % 1) + 1) % 1;
var R = 0.004;
// HIGH from 0 to 0.47, LOW from 0.47 to 0.97, HIGH from 0.97 to 1
if(t < 0.466) return 1;
if(t < 0.466 + R) return 1 – (t – 0.466)/R;
if(t < 0.966) return 0;
if(t < 0.966 + R) return (t – 0.966)/R;
return 1;
}
function ttl2At(t){
t = ((t % 1) + 1) % 1;
var R = 0.004;
if(t < 0.466) return 0;
if(t < 0.466 + R) return (t – 0.466)/R;
if(t < 0.966) return 1;
if(t < 0.966 + R) return 1 – (t – 0.966)/R;
return 0;
}
var CYCLE_MS = 4000;
var WINDOW_MS = 4000;
var start = performance.now();
function laneRect(canvasH, idx){
var pad = 16 * (window.devicePixelRatio||1);
var h = (canvasH – pad*2) / 3;
var top = pad + h*idx;
return { top: top, bot: top + h – 4 };
}
function draw(now){
var w = canvas.width, h = canvas.height;
var DPR = window.devicePixelRatio || 1;
ctx.fillStyle = '#0F0F10';
ctx.fillRect(0,0,w,h);
// grid
ctx.strokeStyle = '#2a2a2a'; ctx.lineWidth = 1;
ctx.beginPath();
for(var i=1;i<10;i++){ var x = Math.round(i*w/10)+0.5; ctx.moveTo(x,0); ctx.lineTo(x,h); }
for(var j=1;j<9;j++){ var y = Math.round(j*h/9)+0.5; ctx.moveTo(0,y); ctx.lineTo(w,y); }
ctx.stroke();
// lane dividers
ctx.strokeStyle = '#2a2a2a';
ctx.beginPath();
for(var k=1;k<3;k++){ var ly = Math.round(k*h/3)+0.5; ctx.moveTo(0,ly); ctx.lineTo(w,ly); }
ctx.stroke();
var elapsed = now – start;
var samples = Math.min(w, 1400);
var fns = [camAt, ttl1At, ttl2At];
var clrs = ['#E6C458','#7BDBD1','#E88BB8'];
var names = ['CH1 CAMERA TRIG','CH2 TTL1 → LED1','CH3 TTL2 → LED2'];
So what does the above look like in real life? Check out this example below! Note that I select in my “lambda” control box the “Stream” checkbox. Then once start capture is pressed, the triggerscope is armed, the camera is readied, and we snag 20 images in < 300ms!
Of course, we can extend this using the 5V DAC control line on the triggerscope for capture of Z as well as channel, and if using the more capable triggerscope 4B, we can control even more devices.
I hope this article has helped explain not only why I make devices such as the triggerscope, but also how it’s important to use software that provides complete control for your light sources and other devices, as well as the ability to do things like this. My sincere thanks to Excelitas for lending me the Edge camera and Tetrem LED, both of which have worked flawlessly on my system. Nebula has drivers for both products fully integrated for streaming control if you have interest in using it.
Thomas Spieker from Opto Gmbh was kind enough to send us a demo assembly of several microscopes made by his company. Opto manufactures a series of highly integrated microscope modules, all provided in a small, integrated form factor.
I was able to work with two different variations. One is a complete transmitted illumination setup with camera and objective at 20x. The second is a reflected light device with integrated LED illuminators.
It’s neat to see such a compact setup on the desk. Using the 20x with a typical gut section stained sample was a piece of cake. Field uniformity on the illuminator was great, and color balance was as expected.
The reflected design unit has both a coaxial and ring-style illuminator position – both are provided in Nebula as control LED options.
Use of the camera was straightforward. It supports 8 and 12 bit readout on a color sensor. I can definitely imagine hooking this up to a small XYZ gantry or even a 3d printer for inspection use. Thanks to Opto for the demo unit!
Here is a video review and live demo of hte unit while running on Starlyte Nebula.
Compact, Simplified TTL and DAC – Micromanager Native
I’m happy to share a new device I’ve been working on after encouragement from numerous clients. This new smaller version of the Triggerscope is intended to bridge the gap between large complex systems and more streamlined setups.
With the Mini you get:
4 TTL Outputs at 5V
2 DAC Outputs at 5V
1 3.3V to 5V input Trigger
Status LEDs for all connections!
A single USB-C Connection to the computer
Full supported Micromanager Triggerscope-Hub Behavior
This was a super fun project for me, as for the first time this board uses a bare MCU, which enabled my team to use any connector we wanted (hence the USB C) as well as full USB-Serial emulation on our own terms.
I’m also quite fond of the baby side-emitting LEDs on the outputs. These LED’s draw almost no current, but having them for all of the outputs makes it so easy to know whether a given control line is on or off. For the DAC LED’s, they change brightness with the output!
I tried to keep this design in-line with the current TG4, as it’s worked well for us over the years. This one is of course thinner, lighter and smaller, but in effect a truly miniaturized version of the larger model.
We had a recent customer asking to solve a rather standard but annoying problem. If I have a manual filter turret, and still want to overlay images, how easy is it to do?
To solve this, we added the ability to drag+drop any image from our capture sidebar into our main view. By adding a demo filter turret to the system config, we captured the real ex, di and em filter settings from the image, insuring metadata is transferred with the images collected. The resulting view is a true multi-dim architecture, saved as an OME set or RGB combined view. Our team made the addition, ran bug testing and edge case validation, pushed the changes and compiled the application in 1 business day.
Being able to react to our customers’ needs in a short timeframe is one of the best parts of working with my team. Nice work!
