Google has applied for a European patent that includes "full-motion video of 360 degrees images, and live feeds from video cameras to provide enhanced driving directions". This seems to suggest that Google has plans for a Street View feature that includes panoramic video.
Google's Street View technology was originally developed by Immersive Media. Immersive Media is a digital imaging company specialising in immersive video. They create panoramic videos that you can pan around inside whilst the video is actually playing. You can see how it works in the video below (just drag the image around to change your view).
Do Google have plans to provide a video version of Street View? Video Street View might not fit so easily with Google Maps as the present still images used in Street View. I'm guessing it would be much more difficult to stitch together a number of immersive videos than it is is to stitch together static images. When a video reaches a road junction the user would have only a very limited time to be able to click on any navigational button to jump to another video street view on the next road.
Google's patent application uses the context of driving directions. Using immersive videos in driving directions does sound possible. If Google had a number of immersive videos of different streets in theory it would be easy to stitch these together for a planned route. So if a Google Maps user wanted driving directions for a particular route it should be possible to fairly seamlessly stitch together a number of immersive videos to create an interactive panoramic video simulation of the actually drive.
This is of course just pure speculation. We will just have to wait and see what Google's real plans are for the patent application.