About me
Welcome to my website where I provide an overview of my scientific work and photography. I am Dutch and currently work as Lead Data Scientist at the Netherlands Labour Authority. Next to this I’m affiliated with Delft University of Technology as an independent researcher. I have a research background in human-computer interaction, visual perception, and computational neuroscience. In my research I use various machine learning techniques to create models of our visual system. I try to understand how we visually estimate properties of objects and stuff around us.
Next to my research I really enjoy photography. I lived two years in Japan where I started using a medium format analog camera, a selection of my work can be found at the photography section.
If you have any questions please don’t hesitate to contact me at mail [at] janjaap [dot] info.
Collective Flow
In 2020 I received a Marie Curie Individual Fellowship for the project ‘Our Elemental Sense of Collective Flow’. In September 2020 I started this project at the Perceptual Intelligence Lab at Delft University of Technology.
Collective flow consists of bodies of individual entities or agents that show both collective and individual behaviours following a coordinated set of rules. There are inanimate occurrences of collective flow (e.g., shaken metallic rods, nematic fluids), microscopic occurrences (e.g., macromolecules, cells, bacteria colonies), and richer manifestations with more intelligent organisms (e.g., insect swarms, flocks of birds, humans).
It is really impressive how we are able to perceive a wide range of behaviours from even very abstract motion patterns. We are able to identify behaviours such as leadership, discipline, and agitation from very basic collective motion patterns. I’m also very interested in future trajectory prediction of collective flow. To test all this I build an online simulator which I can use in experiments.
We developed a wide range of online experiments where different tasks such as ratings, naming, similarity judgements, and ordering tasks were performed. Some experiments allowed for realtime adjustments of the simulations.
We find that we can see very specific behaviour but there is also one very dominant dimension that is linked to the turning rate of the agents which highly influences the grouping and uniformity of the flock. This can be directly measured by the average distance between agents and the average distance in orientation. These metrics explain over 90% of the main perceptual dimension.
This raises the question how we are able to perceive smaller nuances such as an emotional state which have proven challenging to isolate from the main effect of grouping/uniformity.
Optical Flow
Together with Shin’ya Nishida from Kyoto University I investigate the influence of optical flow on perceived motion constancy across different optical material properties.
Motion constancy across different optical properties is a tough computational problem under real-world conditions. Retinal optical flow drastically changes with the optical material properties of the moving object. Specular and diffuse reflections, as well as refractions at object surfaces can produce complex patterns of optical flow that do not correspond with the object motions.
Preliminary results suggest we have some constancy but can only partially compensate for differences in optical flow caused by the different optical material properties. This effect can result in illusions (see video below) where the object motion between objects with different material properties and illumination conditions seem to be different while this is not the case.
Liquids
During my PhD at the department of Experimental Psychology at the Justus-Liebig-Universität Gießen I studied the visual perception of deformable materials. Supervised by Roland Fleming I studied how our visual system estimates the viscosity of liquids. I continued this work with Shin’ya Nishida at NTT Communication Science Laboratories in Japan. Here I researched how we estimate material properties using neural networks that are inspired by our visual system.
The studies
Fluids and other deformable materials have highly mutable shapes, which are visibly influenced by both intrinsic properties (e.g. viscosity, velocity) and extrinsic forces (e.g. gravity, object interactions). How is it possible that we achieve good viscosity constancy despite large differences in the retinal image? In this research I tried to identify the image cues we use to estimate viscosity. We found that mid-level features (e.g. ripples, clumping, spread, piling up) are very important to achieve viscosity constancy across a wide range of contexts. It is hard to quantify these features with image statistics or 3D shape measurements because they can vary in orientation and scale, and occur locally on liquid shapes. Deep learning models can provide additional insights on higher order image-based features that are context invariant.
The stimuli
During my PhD I spend quite some time setting up a technical pipeline to generate stable, precise and realistic liquid stimuli. To be able to do this I followed a three month secondment in Madrid at Next Limit. Here I learned to work with their particle simulation software RealFlow made for the VFX industry. The stimuli were rendered using Maxwell Render. The computational costs of the stimuli are quite high. I wrote specific scripts to distribute the calculations over various systems and clusters. In 2018 I became a certified Realflow Xpert.
Gloss
For my Master degree I graduated on a research project studying gloss perception in the Perceptual Intelligence Lab at Delft University of Technology with Sylvia Pont and Maarten Wijntjes.
The study
We investigated the influence of the spatial structure of the illumination on gloss perception. The inspiration came from various art works like the paintings of Vermeer, where much simpler highlight shapes are used to depict real world situations. We find that more complex highlight shapes were perceived to produce a less glossy appearance than simple highlight shapes such as a disk or square.
The stimuli
A diffuse light box with differently shaped masks were used to produce a set of 6 highlight shapes. In the box we placed spherical stimuli that were painted in 6 degrees of glossiness. This resulted in a stimulus set of 6 highlight shapes and 6 gloss levels, a total of 36 stimuli. Observers were asked to rate glossiness looking using the real stimuli and photographs displayed on a monitor.
Publications
Papers
2020 Jan Jaap R. van Assen, Shin’ya Nishida, & Roland W. Fleming. Visual perception of liquids: Insights from deep neural networks. PLoS Computational Biology 16(8): e1008018. doi:10.1371/journal.pcbi.1008018. [PDF]
2018 Jan Jaap R. van Assen, Pascal Barla, & Roland W. Fleming. Visual features in the perception of liquids. Current Biology, 28(3), 452-458, doi:10.1016/j.cub.2017.12.037. [PDF]
2017 Filipp Schmidt, Vivian C. Paulun, Jan Jaap R. van Assen, & Roland W. Fleming. Inferring the stiffness of unfamiliar objects from optical, shape, and motion cues. Journal of vision, 17(3), 18-18, doi:10.1167/17.3.18. [PDF]
2017 Vivian C. Paulun, Filipp Schmidt, Jan Jaap R. van Assen, & Roland W. Fleming. Shape, motion, and optical cues to stiffness of elastic objects. Journal of vision, 17(1), 20-20, doi:10.1167/17.1.20. [PDF]
2016 Jan Jaap R. van Assen, & Roland W. Fleming. Influence of optical material properties on the perception of liquids. Journal of vision, 16(15), 12-12, doi:10.1167/16.15.12. [PDF]
2016 Jan Jaap R. van Assen, Maarten W. A. Wijntjes, & Sylvia C. Pont. Highlight shapes and perception of gloss for real and photographed objects. Journal of Vision, 16(6):6, 1–14, doi:10.1167/16.6.6. [PDF]
Please contact me for a full CV at mail [at] janjaap [dot] info or by LinkedIn.
Photography
In Japan I switched to analog photography using a medium format camera (Mamiya 7). Below you will find a selection of my photos.
More work can be found on Behance. I recently switched to Behance and I will gradually add more work.
