When computers look at art
Recent triumphs and future opportunities for computer-assisted connoisseurship of fine art paintings and drawings
David G. Stork
Abstract: Our cultural patrimony of fine art paintings and drawings comprise some of the most important, memorable, and consequential images ever created, and present numerous problems in art history and the interpretation of “authored” stylized images. While sophisticated imaging (by numerous methods) has long been a mainstay in museum curation and conservation, it is only in the past few years that true image analysis—powered by computer vision, machine learning, and artificial intelligence—have been applied to fine art images. Fine art paintings differ in numerous ways from the traditional photographs, videos, and medical images that have commanded the attention of most experts up to now: such paintings vary extensively in style, content, non-realistic conventions, and especially intended meaning.
Rigorous computer methods have outperformed even seasoned connoisseurs on several tasks in the image understanding of art, and have provided new insights and settled deep disputes in art history. Additionally, the classes of problems in art analysis, particularly those centered on inferring meaning from images, are forcing computer experts to develop new algorithms and concepts in artificial intelligence.
This talk, profusely illustrated with fine art images and computer analyses, argues for the new discipline of computer-assisted connoisseurship, a merger of humanist and scientific approaches to image understanding. Such work will continue to be embraced by art scholars, and addresses new grand challenges in artificial intelligence.
Bio: David G. Stork, PhD, is Adjunct Professor at Stanford University and a graduate in Physics from MIT and the University of Maryland; he also studied Art History at Wellesley College. He has held faculty positions in Physics, Mathematics, Computer Science, Statistics, Electrical Engineering, Neuroscience, Psychology, Computational Mathematical Engineering, Symbolic Systems, and Art and Art History variously at Wellesley and Swarthmore Colleges, Clark, Boston, and Stanford Universities, and the Technical University of Vienna. He was a 2023 Leonardo@Djerassi Fellow, is a Fellow of seven international societies, and has published eight books, 220+ scholarly articles, and 64 US patents. His Pixels & paintings: Foundations of computer-assisted connoisseurship (Wiley) appeared last year and he is completing Principled art authentication: A probabilistic foundation for representing and reasoning under uncertainty.
From Newton to Latour – Towards Physically and Socially Plausible 3D Generation
Jingyi Yu
Abstract: Recent advances in 3D generation have significantly pushed the boundaries of visual realism, enabling the creation of highly detailed virtual environments and objects. However, despite these strides, current systems still face substantial limitations in producing physically plausible interactions, especially when multiple objects or parts within a scene engage with one another. These limitations manifest in unrealistic motion, collision detection errors, and a lack of true inter-object dynamics, which undermine the fidelity of 3D simulations. Furthermore, most 3D generation systems fail to address the social dimensions of these environments, neglecting the implications of human and non-human actor interactions in a way that resonates with real-world complexity. In this talk, I show that 3D generation must move beyond mere visual realism and embrace both physical plausibility and social meaning. Drawing on Bruno Latour’s Actor-Network Theory (ANT), we propose a framework that integrates both physical laws and social constructs into the 3D generation process, allowing for richer, more meaningful representations of interactive spaces. Our recent efforts focus on embedding these dual principles—physics and social interaction—into generative models, providing a more holistic approach to creating 3D environments that are both physically coherent and socially relevant.
Bio: Jingyi Yu is an OSA Fellow, IEEE Fellow and an ACM Distinguished Scientist, Director of the MoE Key Lab of Intelligent Perception and Human-Machine Collaboration. He received B.S. with honor from Caltech in 2000 in Computer Science and Applied Mathematics and Ph.D. from MIT in EECS in 2005. He is the Inaugural Chair Professor of the ShanghaiTech University. He also serves as the Vice Provost of the university and the Dean of the School of Information Science and Technology. Dr. Yu has been working extensively on computational imaging, computer vision, computer graphics, and bioinformatics. He received both the NSF CAREER Award, Air Force Young Investigator Award, Magnolia Memorial Award. He has over 10 PCT patents on AI-driven computational imaging solutions, many of which have been widely deployed in smart cities, digital human, human-computer interactions, etc. He has served as an Associate Editor of IEEE TPAMI, IEEE TIP, and Elsevier CVIU as well as program chairs of several top AI conferences including ICCP 2016, ICPR 2020, WACV 2021, IEEE CVPR 2021, and ICCV 2025. He is also a member of the World Economic Forum’s Global Future Council, serving as a Curator of the Metaverse Transformation Map.