Validation and Improvement of Algorithms in Computer Graphics, Computer Vision, Visualization

What information is available at a glance from a scene? Can we create computer vision algorithms that capture the “gist” of a scene?

Humans are able to get the “gist” of a scene already after 150ms. Using perceptual experiments, we have shown that humans can estimate the horizon of a scene with high accuracy. First computational experiments indicate that this might be done using a global, frequency spectrum analysis (Herdtweck and Wallraven, 2010, 2013).

  1. C. Wallraven, B. Caputo, and A.B.A. Graf. Recognition with Local Features: the Kernel Recipe. In ICCV 2003, volume 2, pages 257–264. IEEE Press, 2003.

  2. C. Herdtweck and C. Wallraven: Beyond the horizon: perceptual and computational estimates of horizon position. Applied Perception, Graphics, and Visualization (2010)

  3. C. Herdtweck and C. Wallraven. Estimation of the horizon in photographed outdoor scenes by human and machine. PLoS One, pages 0-0 (2013).

  4. C. Wallraven and J. Freese. We remember what we like?: Aesthetic value and memorability for photos and artworks - a combined behavioral and computational study. Journal of Vision, 2015.

  5. C. Wallraven, R. Fleming, D. Cunningham, J. Rigau, M. Feixas and M. Sbert: Categorizing art: Comparing humans and computers. Computers and Graphics 33(4), 484-495, 2009.

  6. J. Rigau, M. Feixas, M. Sbert and C. Wallraven: Toward Auvers Period: Evolution of van Gogh's Style. Computational Aesthetics 2010.

  7. J. Fischer, D. Cunningham, D. Bartz, C. Wallraven, H. Bülthoff, and W. Strasser. Measuring the discernability of virtual objects in conventional and stylized augmented reality. In 12th Eurographics Symposium on Virtual Environments, pages 53–61, 05 2006.

  8. T. Stich, C. Linz, C. Wallraven, D. Cunningham, and M. Magnor. Perception-motivated interpolation of image sequences. ACM Transactions on Applied Perception, 02 2010.

  9. J. Wu, R. Martin, P. Rosin, X. Sun, Y. Lai, Y. Liu, and C Wallraven. Use of non- photorealistic rendering and photometric stereo in making bas-reliefs from photographs. Graphical Models, 2014.

Kernelizing feature matching

In my PhD work, Arnulf Graf, Barbara Caputo and I came up with an easy extension to the Support Vector framework that would allow people to perform feature matching inside of the Kernel evaluation. Although our initial proof needed to be revised, this publication was the first to connect local feature methods with Kernel methods and to demonstrate significant gains in performance (Wallraven et al., 2003)!

What information determines aesthetic judgments? Can a computer become an art critic?

With the recent advances in image processing, we ask how far state-of-the-art image measures can be used to model the aesthetic experience of observers looking at a painting. In a series of papers, we have shown that these measures already capture aspects of aesthetic judgments (e.g., Wallraven et al., 2009, Rigau et al., 2010) - the computer, however, will need much more training to become an art expert!

In another project, we have shown that although art is seen as more aesthetic than ordinary photographs, they are actually not remembered better. Several computational features were used to try to model memorability of either category, but also here we found that low-level features have limited power for prediction (Wallraven et al., 2015).

Evaluation of algorithms

In a visual ”Turing test”, we found that people were unable to tell the difference between computer graphics objects that were inserted into the scene and real objects, thus validating the approach.

Similarly, we were able to evaluate a sophisticated image interpolation technique using perceptual experiments to show that it produced the least perceptual artefacts.

In addition, several algorithms on creation of bas-reliefs were evaluated to see which methods would produce the most appealing effect.