Video Summary, Browsing, Search

Video Summary

Invented and developed several approaches for summarizing video by selecting appropriate keyframes and arranging them in layouts such as Manga comic books or stained-glass collages.

Technical challenges. Designed and contributed to several algorithms that perform video and audio analysis to segment a video and to select the number of keyframes needed by the application. Contributed to the initial Manga video summary invention and invented an improved layout algorithm that quickly fills all available space. Wrote versions using Java applets and timelines painted in HTML canvases filled combined with image placement in JavaScript. Implemented a client-server architecture that performed the layout on the server.

Technologies. Java, JavaScript, HTML Canvas, HTML Video.

P. Chiu, A. Girgensohn, Q. Liu. Stained-Glass Visualization for Highly Condensed Video Summaries. IEEE International Conference on Multimedia and Expo (ICME), 3, 2059-2062, doi:10.1109/ICME.2004.1394670, 2004.
A. Girgensohn. A Fast Layout Algorithm for Visual Video Summaries. IEEE International Conference on Multimedia and Expo, v. II, 77-80, doi:10.1109/ICME.2003.1221557, 2003.
A. Girgensohn, J. Boreczky, L. Wilcox. Keyframe-Based User Interfaces for Digital Video. IEEE Computer, 34(9), 61-67, doi:10.1109/2.947093, 2001.
J. Boreczky, A. Girgensohn, G. Golovchinsky, S. Uchihashi. An Interactive Comic Book Presentation for Exploring Video. CHI, 185-192, doi:10.1145/332040.332428, 2000.
A. Girgensohn, J. Boreczky. Time-Constrained Keyframe Selection Technique. Multimedia Tools and Applications, 11(3), 347-358, doi:10.1023/A:1009630817712, 2000.
P. Chiu, A. Girgensohn, W. Polak, E. Rieffel, L. Wilcox. A Genetic Algorithm for Video Segmentation and Summarization. IEEE International Conference on Multimedia and Expo, vol. III, 1329-1332, doi:10.1109/ICME.2000.871011, 2000.
P. Chiu, A. Girgensohn, W. Polak, E. Rieffel, L. Wilcox, F. Bennett. A Genetic Segmentation Algorithm for Image Data Streams and Video. Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers, 666-673, 2000.
A. Girgensohn, J. Boreczky, L. Wilcox, J. Foote. Facilitating Video Access by Visualizing Automatic Analysis. Human-Computer Interaction INTERACT, 205-212, 1999.
A. Girgensohn, J. Foote. Video Classification Using Transform Coefficients. International Conference on Acoustics, Speech, and Signal Processing, vol. 6, 3045-3048, doi:10.1109/ICASSP.1999.757483, 1999.
S. Uchihashi, J. Foote, A. Girgensohn, J. Boreczky. Video Manga: Generating Semantically Meaningful Video Summaries. ACM Multimedia, 383-392, doi:10.1145/319463.319654, 1999.

Video Browsing

Developed approaches for browsing through collections of videos and within a single video. Created several keyframe slider approaches that let users quickly locate interesting parts in videos. This work formed the basis of a Fuji Xerox product for managing lecture video collections.

Technical challenges. Contributed to analysis software for video segmentation and keyframe selection. Developed a web-based system that displays a file-system hierarchy of videos including a Java applet for each video that visualizes changes and other temporal data and controls a keyframe display via JavaScript. Created a helper application that launched a C++ app with an embedded Windows Media Player for playing the video from a shared file system (1998 technology) and controlling playback by skipping to interesting parts of the video. Created a video keyframe slider that uses a combination of HTML video and canvas to control video playback with keyframes attached to a timeline.

Technologies. C++, Windows Media, Microsoft SMB, Java, JavaScript, Common Gateway Interface (CGI).

J. Boreczky, J. Foote, A. Girgensohn, L. Wilcox. Interactive Similarity Search for Video Browsing and Retrieval. RIAO Conference, Content-Based Multimedia Information Access, 637-648, 2000.
L. Wilcox, A. Girgensohn, J. Foote, J. Boreczky. MBase: Indexing, Browsing and Playback of Media at FXPAL. ACM Multimedia (Part 2), 205, doi:10.1145/319878.319942, 1999.
J. Foote, J. Boreczky, A. Girgensohn, L. Wilcox. An Intelligent Media Browser Using Automatic Multimodal Analysis. ACM Multimedia, 375-380, doi:10.1145/290747.290804, 1998.

Video Search

Created algorithms and a search UI to successfully participate in the TRECVID evaluations.

Technical challenges. Contributed to the algorithms for analyzing the videos to be searched. Designed and developed a Java app that provides a UI for the search by visualizing matches in keyframe collages and in the timeline for the active video. Combined visual and text search.

Technologies. Java Swing, QuickTime, C++, DirectShow.

A. Girgensohn, J. Adcock, M. Cooper, L. Wilcox. A Synergistic Approach to Efficient Interactive Video Retrieval. INTERACT, LNCS 3585, 781-794, doi:10.1007/11555261_62, 2005.
A. Girgensohn, J. Adcock, M. Cooper, L. Wilcox. Interactive Search in Large Video Collections. CHI Extended Abstracts, 1395-1398, doi:10.1145/1056808.1056925, 2005.
J. Adcock, A. Girgensohn, M. Cooper, T. Liu, E. Rieffel, L. Wilcox. FXPAL Experiments for TRECVID 2004. TRECVID, 2004.