PHOTOOG Photography writings by Olivier Giroux

27Jan/09Off

D700 Shoots Video

UPDATE: http://ogiroux.wordpress.com/2009/02/26/your-d700-shoots-video-too/

A couple of nights ago I saw this post on CanonRumors. I thought to myself… that’s so obvious! I filed the thought away after that, and planned to forget about it.

The next day I noticed a wave of news about updated remote-control software by a 3rd party (i.e. not Nikon).  That got me thinking... clearly there is an SDK for Nikon cameras.  How hard is it to get?  I wondered.

Not hard at all.  Basic form-filling skills is all you need.  Last night I downloaded out the D700 SDK. Minutes later I had built the sample remote program and was pulling my D700’s strings over USB. That includes control of the Live View feed.

Good news: it is a viable video source. It’s fast enough, and the quality is sufficient.

Bad news: it’s a bit too low-quality to be really exciting. It’s roughly 30% below 480p resolution.  The most unfortunate thing is they create the Live-View image by decimating the sensor data rather than downsampling it – as a result it aliases, moirés and looks terrible in low light.

That's it for the bad news though.  The feed travels over the wire at 100fps (I measured), within which maybe 30fps' worth are unique frames.  Each frame is basically a NEF embedded thumbnail, each one is a fully-formed high quality JPEG file.  The result has the potential to look as good (or bad) as a DVD, roughly.

I spent about 4 hours total coding up a video software prototype from the SDK sample. Here are the preliminary results, shot this morning.  I intended it to be a single short piece but my choice of codec for the prototype tied my hands a little bit.

[youtube=http://www.youtube.com/watch?v=Z5LD-t78KP8]
D700 and Zeiss 2/100 Makro-Planar T* at f/4.

[youtube=http://www.youtube.com/watch?v=9ZhK84ZTr5o]
D700 and Zeiss 2/35 Distagon T* at f/2.8.

Next I will try to "productize" the software (which is to say, make it look usable by people-not-me) and pick a more convenient codec.


Comments (97) Trackbacks (13)
  1. That is totally awesome. Well done! How did you get the SDK? I’m a coder myself and would also like to have a go at it!

  2. Very cool. I am also a software developer. Where did you get the SDK?

    Thanks!

  3. Hey Olivier, I’m your neighbour in Bedford. This is really awesome! If you can create a web-interface for this you could definitely use it as a baby monitor or take pictures say every 30 mins and create a stop motion video. Would be great for astro-photography. Well done! If and when I have a kid, I could convince my wife to upgrade from our D200 now. 😀

    • I think the baby would be bothered by the sound of the mirror flipping up and down all the time. It’s not so discreet.

      You wouldn’t need to do much C/C++ hacking for the kind of automation you describe. The SDK sample exposes the entire API to the command-line already. You could even implement my video feature using nothing but a Perl script, if you like Perl.

    • Well, for this you could just use the built in Interval Time Shooting and put the camera in Thethered mode with a computer, say with lightroom 3. Then you could easily put the images on the web automatically.

  4. I can’t wait to see a “usable” software. This is not ideal, but that can be usefull.
    I guess it will work the same with the D3 ?

    • Yes, the SDK appears to be mostly the same between the D300/D700/D3. I’m going to guess that the same source code (though perhaps not the same binary) will work with all LiveView-capable Nikon bodies.

      As for ‘usable’… the biggest issue is the kind of PC you might need. I’m aiming for a solution that could work with a 299$ netbook – minimizing size, weight and cost. Duck-tape one of these to your tripod legs and you’re good to go. 😉

  5. Really nice to know that you have that in mind too !!
    No way i carry my big pc tower on the field 😀
    (well i use macs, but i can’t ask you to do a mac version of the software yet, i know how much it represents….)

    • I won’t be making a Mac version only because I don’t have easy access to one. If I did, I would. If Apple decides to jump into the netbook arena, that would certainly help.

  6. bah… copycats.

    kidding aside, well done.

  7. just to be able to do 480p on my D3 would be great, one less thing I would have to carry with me! well done!

  8. Hi!

    It’s great what you made! Is it possible that you published a source code (eg. on GPL license)? It would be amazing!

    With kissing
    Ann

  9. Hi!

    Great job! Is it possible that you published a source code (eg. on GPL license)? It will be great!

    Ann!

  10. Very sweet. I’ll deffinitely be watching for this.

  11. Any chance you can post (or email me) the code? I’ve got the SDK and would love to tinker the app you’ve got – even if it is alpha quality.

    Thanks,

    Tyler

    • I need to decide if some (or all?) of my code has licensing issues first. I may not release code until after I release a binary (but I still plan to release the code).

  12. I assume the same kind of restrictions as with the D90 apply in terms of shooting duration with live view in order to stop the sensor overheating?

    • Yes the same restrictions apply. Actually the time permitted is exactly the amount of time which the D700 allows live-view to be engaged. I don’t have the exact figure but it flips the mirror up eventually after a few minutes.

      I think this restriction is here to stay, even if it is variably longer in future cameras.

  13. Great work! This setup would also be really useful for doing at home fun time-lapse video projects. I hope you build support for that into your app. Thanks.

  14. Vcx, you don’t need this for time-lapse movies. The D700 has a built-in intervalometer you can use for this.

  15. I think the D200 has an intervalometer as well, so Saket will have to hide that little detail from his wife …

  16. Is there such a thing for canon?

  17. Hey

    That’s very interesting! I’m an IT student, freshman, so I’m not that good.
    You did it in High Level? I expected it to be something more like assembly.
    What about the interface?
    Do you also get a compiler?
    Does it also have something like a manual?

    Emil.

    • You may be confused: my program runs on a PC, not inside the camera. It was in high level code, yes. It could be done in just about any language that can interact with a console (though that is not what I did). Compiler-wise, a regular compiler for Windows will do, you can get these for free from a dozen places including Microsoft.

      There is a manual, but it is very poorly translated. It definitely wasn’t written by a native english-speaker. The code sample is all the documentation that’s really needed.

  18. well, i didnt know that – thanks for sharing

  19. So you are running a program with the camera connected to PC? I don’t quite get it..

  20. Emil, yes the camera is connected to a PC running my capture program.

  21. cool post, have had the D700 since it came out but always learning new ways to use it, thanks.

  22. Oh. So the SDK is from a program that does ‘thethered shooting’?

  23. It’s not from one, it is used to make one.

  24. SWEET!!! This is freakin’ awesome! Great work so far and as a fellow D700 owner that wants some video capture love, I will be eagerly checking your blog for updates! Keep up the excellent work!

  25. Could we perhaps get this to the D90 aswell? The video we’re getting from it atm aint that good so something better would be really nice!

    • While I’m sure it would work with the D90 as well… no matter what quality you’re seeing with the D90, this is surely worse. Even if the D90 used the same basic recipe to produce lower-resolution frames, the fact that it outputs 720p video natively puts it head-and-shoulders above the “below-480p” results I’m getting. I suspect they have also applied a software high-frequency AA filter to reduce the artifacts.

      At best, the Live-View feed would contain the exact same frames that the D90 saves to MJPEG files. The MJPEG movie file is just a bunch of JPEGs glued together, and I’m willing to bet that their the same JPEGs that Live-View gets given that they are fully-formed, etc… and have the PictureStyle options applied to them already. Sorry, the D90 already does better and probably couldn’t be improved without new hardware.

  26. Olivier, this is great reading, very interesting stuff 🙂

    When you release the software, will you be integrating regular camera control software into it? I’m aware that you want to try and keep size/complexity down in order to allow lightweight running, but even a basic implementation of the camera controls (live view, AF operation, shutter/aperture/iso and picture control settings) would make the software a really useful all-round package.

    Also, you’ll need to forgive my ignorance on this one, but is there any way of accessing the D700’s HDMI output feed to be sent down the USB cable? I’m just thinking if the camera CAN do live view in HD, that it may be tappable in some funny way. Of course, as you can probably tell, I’m certainly no software expert!

  27. I don’t plan to integrate control features. The SDK is free to download, it’s pretty easy to use, and I’m fine with letting other people build that application for the time being. My end-goal is a bare-bones electronic viewfinder with a record button, which hopefully writes MJPEG files at full speed without need for a codec (if I can find time to sift through the 100+ page OpenDML spec jointly with the sample files/code I put my hands on…).

    You know… I hadn’t realized that the D700 outputs HD-resolution LiveView via HDMI… the connector is a “mini” variant so none of my A/V cables at home will fit the camera obviously. That’s very interesting. So the D700 is actually capable of producing good-quality video internally.

    I will dig around the SDK, but my hopes are faint. I think that in order to record that feed you’d need a DVI frame-grabber and hope the HDMI connection isn’t actually encrypted (which given DpReview’s recordings, suggest is doable). Thanks for the tip though.

  28. Olivier, I have followed your “expose'” on this camera with great interest and wonder if this will work with the D300.
    Thanks, FS

  29. Now my question is, can a D200 be adapted to utilize LiveView? I can’t afford any new gear at the moment but I’m desperate to shoot HD video. I may have to rent gear for my next project.

    I love my D200 but I’ve pushed it as hard as I can and now need to upgrade to an HD capable machine.

    • Paul, that would be very surprising. LiveView nominally requires the ability to scan pixels off the sensor at a higher frequency than normal, which implies physical datapaths built into the hardware. It also means the image processor needs to process these faster, which means more datapaths and computing power built into the hardware.
      I am certain that if you are patient for another couple of months (say, 4-10 months), you will get a “D400” (hypothetical) camera that shoots HD video roughly as well as the 5DmkII does.

  30. Nice work Oliver! I have been telling Nikon they should forget this business of obsoleting our hardware and start selling software upgrades to provide features like video. Would you rather pay $100-200 for a software download to activate a feature or pay $2000 for a new camera? I was not happy after spending $1600 on my D300 to find out this little D90 camera had video. I am not going to turn around and purchase a Nikon D400 – Nikon should figure out how to enter the software market! http://snurl.com/aws50

    In the meantime, Oliver, keep up the great work and give us this feature sooner! Keep shooting, John

  31. Great work Olivier!

    Immediately after the release of the D90 I asked Nikon for adding video to the D700 too. “It’s technically not possible” was their reply”. 🙁

  32. This is amazing. I’d really like to use it with my D60. Is this possible?

  33. It was my impression that the LiveView frame rate of the D700 is only 15 fps therefore I didn’t doubt Nikons answer. But you proof that’s not the case.

    Some people (me too) tried to capture the LiveView display in Nikons Camera Control Pro with a screen capture program. It works but I only got 12 fps. The 640×480 video didn’t look too bad just like your examples but it included an annoying focus rectangle overlay. The colors and low depth of field are fantastic though.

    I’m looking forward to your little program with LiveView window and a record button that stores the frames without recompressing them. If it has just a record button I’m fine too. Do you plan to record 24 or 30 fps?

  34. You are a genious…
    I wanna try it the first possible !

  35. Wow, they really improved the SDK acquisition process. It used to be paper forms that you had to physically mail to them, which they actually reviewed, and they rejected my last application (for an art installation project).

    Thanks for bringing this back to mind, I’ve been writing control software via the Windows Image Acquisition DLL, and it’s aaaawwwwwwful.

  36. Any idea when there will be a downloadable solution? Can’t wait!

  37. hehe I wondered when someone else would do it 🙂

    I did this a while ago:
    http://www.mountainstorm.co.uk/data/cat-test/cat-test.mov

    btw the lighting in my room is terrible.

    If your interested you can follow my Nikon tethering applications development here (page 4 has stuff about liveview video):
    http://www.lightroomforums.net/showthread.php?t=3020&page=7

    Its not tied to Lightroom (my apps called StdioTether) it just happens to be the main reason I developed it. I’ve got a new beta out which supports liveview … give it a go, Im going to add recording it once I’m happy everything else is working.

    I am slightly interested though at your frame rate. I found my D300 had a framerate of ~10fps. Now the D700 may just be faster but I’m intrigued. I take it you are just polling deviceready and when it is downloading the liveview image .. as fast as you can?

    Maybe I should have another look at it. Good work though

    Rich

  38. Well after looking again at the Nikon docs I realized I was doing something wrong. A quick tweak later and it appears you are indeed correct, well over 60fps from a Nikon D300’s liveview.

    The only issue I can see is that its a bit … erm, bendy. If you pan around you get the walls bending.

    Oh well, its free so I can’t complain

    Rich

  39. Is the recording limited to # of minutes like that of a D90?

    *Sorry if this was asked before, I skimmed through and don’t seem to see any info on this. 🙂 *

  40. Thanks for reminding me that the SDK is avail
    and the hint that it is easy to get (know).
    Tried to get it some years ago!
    The code would be nice. And now a true need
    for learning C++!

  41. I am extremely excited about this opportunity. Thanks for shedding some new light on my already amazing camera. I’d be more then willing to Beta test if you need. BTW, if you need any hi-speed tests, let me know – I work with race horses.

    Thanks
    Jim

  42. I’m thinking about investing in a Blackmagic Intensity HDMI capture card for my Mac to capture my D90s LiveView feed via HDMI.

    Do you think a hack could be written to increase the resolution of the video feed to at least 1080p?

    • Capturing HDMI would be interesting, and would definitely deliver better resolution than what I’m doing now. Now don’t confuse the scope of the “hacks” you can write. You can’t easily go and modify the camera’s own internal behavior like that. I didn’t write a hack, all I did was introduce two legitimate pieces of software to each other.

  43. Difficult, indeed. Usually requires firmware mods, which can be risky sometimes. It’s fun to look though!

  44. Hi Olivier!
    I used the same technique at d300 and the result is quite all right – 640×426@25fps. One thing which would be quite interesting would be to connect the camera to some iphone/itouch and control the video recording from there. Any plans of doing this?

  45. Does anyone think it would be possible to create a power focus system for the D700? A toggle system that would roll focus in or out overriding or replacing autofocus function even if it required an external switch, actually, that would be ideal

  46. I tried to tweak the sample SDK program to fetch higher resolution liveview images with better interpolation.. but the evailable API poses a limitation. E.g.: I cannot even request images bigger than 640×480 pixels when in liveview mode.. it just returns an error. Still on it though.. my primary interest is in being able to interpolate from a higher resolution image when in liveview.

  47. In the pervious comment I posted, read it about D90.. not D700.

  48. Hi again,
    I followed your technique and prepared a program, which can capture video on the D300. It is a very very simple one but it should do the job 🙂 Thanks again for the information!
    http://d300.rdx.cz

  49. I just tried building a simple video recording test app with the Nikon sdk on mac os x. It works pretty well if I poll for live view images at 24fps. For me grabbing an image takes 20-30ms. Does anybody know if there Is any way to lock the exposure while in live view mode? I had no success doing it programatically, and I also tried setting the camera to manual exposure before entering live view. Nothing seems to work.

    • Hi! How are you progressing with your macintosh version? I am desperate to get this kind of functionality onto my mac for a D300… is there any chance you will distribute your program?

      thanks
      -Max

  50. Thanks a lot for the program!
    Anyone who has bookmarked this page should look at
    http://ogiroux.wordpress.com/2009/02/26/your-d700-shoots-video-too/

  51. That’s interesting.

    so. meantioned that there are about 100 frames per second squeezed through – is it actually possible to capture them all ? like the work-around with d300-video-bin and ramdisk, it should be possible to get all the jpegs…

    can you make it possible to select the framerate ? i would like to get all the frames, that are coming in.

  52. uhm ok. so why go for 60fps, MountainStorm ?

  53. I settled on 60fps as its 2x 30fps … which is a nice safe persistence of vision speed. Thus even with stutter and other effects you stand a good chance of getting a smooth video.

    I’ve actually turned it back down to ~15fps to using stupid amounts of CPU and dropped the idea of recording it. Other people appear to be writing apps specifically to do that (which is easier anyway) and on a D300 (at least) you can see the distortion from the rolling shutter (its why the cameras still in all the videos you see).

    Unless someone has some cool software to repair the rolling shutter effect (which should be possible) your not going to make a film with a D300.

    Rich

  54. For anti-rolling-shutter you can use virtualdub and the deshaker plugin. it has the capability to correct wobble and shaking plus eliminates the rolling shutter effect.
    (via avisynth one may script the incoming frames to be corrected.)

  55. Cool … now if someone can name a OS X program which does it (and does it well) I might be tempted to give it a go again.

    Rich

  56. there’s parallels.. at least for testing purposes. and if you want to implement it in your program, forget it. it takes too long and at least 2 passes, so no chance to get 15 fps in realtime i think (?)

  57. Hi!
    Great Prog!!

    Are there any news about the HDMI output?? Would be so great if its possible!! But the first question is…sends the d700 a video over the hdmi-out or is there just a pictureoutput? Can someone connect the d700 over hdmi with a tv and check this out?? cant do it here because i dont have any hdmi interface on my tv ;(

  58. SDK acquisition process has really been improved. I like the progress…

  59. hi olivier!
    i’m playing with the terminal window, is it possible to expand the getLiveViewImage command?
    i’m not a developer, as u can see 😉

    greetz

    • What do you mean by “expand the getLiveViewImage command”? The image can’t be made larger, if that’s what you’re asking.

      • no sorr’, i know i can adjust the size and compression, but just because i got single pictures, and i wonder how to get a 24fps in a row-movie .. there is no extra command like this?

  60. Mai, the application provided by Nikon doesn’t do movies. That’s why I spent several nights of free time to make nkvid. Just go download nkvid if that’s what you want.

  61. Hi,

    I am in need of doing long recordings which has not been possible with other dslr since there is usually an inbuilt limit. Im wondering if this method will allow me to record long videos approximately 20-30 mins

  62. I have a D700 and would like to get this working, but for some reason it won’t run on my computer. I figure this is likely because I’m running Windows 7 64bit. Can anyone else confirm this is the problem?

    thanks,

    Tim

    • Dear Mr T,

      I did develop this on Vista 64-bit, and I don’t expect Windows 7 changed anything that matters in this area.

      Try to run it with administrator privileges to see if it helps.

      Cheers,

      Olivier

      • Thanks for your reply, but this still is not working.
        I have tried switching USB ports but have still failed to run the software. I know the computer is seeing the D700 and I even get the little camera icon on the Windows toolbar when the camera is plugged in and turned on. I’m at a loss for what the problem could be. I’ll keep trying and let you know if I figure it out.

        thanks,

        Tim

  63. Hi Oliver,

    I tried doing this with video, recording audio at the same time via Audacity.

    Now, the funny thing is that 20 seconds of shooting the atomic clock online produces a 25 second video. So, what exactly is going on here?

    Thanks,
    Ryan

  64. Thanks a lot for the program!

  65. Thanks a lot for the program! In the pervious comment I posted, read it about D90.. not D700.

  66. SDK acquisition process has really been improved. I like the progress…

  67. Hi Olivier
    Many thanks for the software. Is it possible modify the software for D3s camera and saving some uncompresed image sequence like RAW, tiff or jpg (100% quality) format? I’d like it because the default MJPEG in D3s is not that great for editing and color grading.

    Thanks

    • In theory I could update the software to support newer cameras simply by recompiling it with the latest SDK, but that wouldn’t help you improve video quality. The SDK merely lets the software peek at the live-view image at the same or lower quality than the D3S records on its own.