Thanks for the link Ned, I will check it out
Yeah I think USB based serial has to be the starting point because it keeps things very simple. In the last few days I have managed to get an Arduino moving servos in time to a script being played on a computer via a serial protocol I have devised. I encoded a short video clip in 3-axes using a CSV file and was able to play it back with the video and servos (sort of) synchronised.
The protocol is inspired by G-code and based on buttplug’s generic device commands. It takes the form of ASCII character strings.
Linear move, device 2, position 80 (0.8), ramp at speed 500 (0.5/sec).
The commands can be shortened to mean instant reaction for live control, ie “L080” or “L80” (set device 0 to 0.8). Can substitute V or R for L to control vibration or rotation devices, and I instead of S to specify a ramp time interval rather than a ramp rate.
My Arduino code reads the serial commands and translates them into the appropriate servo responses. On the computer end I am using a basic python script to send the commands (and I mean extremely basic - Sunday was the first time I ever used python).
Will try to post pictures later, and happy to share my code.