Sloppiness and Parameter Identifiability, Information Geometry by Mark Transtrum

1,350 views · Published 2 January 2017 · 1:30:54 · Indexed 6 November 2026

Channel: International Centre for Theoretical Sciences · 2017 · Science & Technology

Watch on YouTube

26 December 2016 to 07 January 2017

VENUE: Madhava Lecture Hall, ICTS Bangalore

Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical and quantum systems can be characterized in terms of constraint satisfaction problems, with the physical behavior of these theories – for example, glassy behavior – governed by the computational complexity of such problems. Information theory also provides a language for laying the foundations of nonequilibrium statistical mechanics, and plays a central role in understanding the emergence of a small number of useful parameters in models of physical and biological phenomena.

This school, organized jointly by the ICTS and Brandeis University, will cover a variety of forefront research topics in physics and biology, in which information and computation play a central role:

• Thermodynamics of information processing
• Coding and computation in statistical mechanics
• Quantum information in extended systems
• Information geometry and ”sloppy” models

Inspired by the history of the renormalization group, the goal is to provide an interdisciplinary survey of the uses of information theory in physics and biology, in order to seed future conceptual breakthroughs in our understanding of complex physical and biological systems. The school will consist of a set of pedagogical lectures by prominent researchers in statistical physics, biological physics, and quantum information theory. These will be supplemented with tutorials led by the organizers and structured around worked problems and selected readings of key papers. The pedagogical activities will be followed up with seminars by the lecturers on current research.

The school is intended for advanced graduate students, post-docs and other junior researchers


CONTACT US:
[email protected]

PROGRAM LINK
https://www.icts.res.in/program/INFOASI

Table of Contents (powered by https://videoken.com)
0:00:00 US-INDIA ADVANCED STUDIES INSTITTE: CLASSICAL AND QUANTUM INFORMATION 
0:00:08 SLOPPINESS AND PARAMETER IDENTIFIABILITY, INFORMATION GEOMETRY, AND THE ROLE OF EXPERIMENTAL DESIGN (LECTURE 1)
0:00:11 INFORMATION GEOMETRY AND SLOPPY MODELS
0:00:28 ABOUT ME
0:02:06 OUTLINE
0:02:37 THE BIG PICTURE: MATHEMATICAL MODELING IN SCIENCE
0:03:11 REFERENCES
0:04:37 KEY OBSERVATION: THE MAP FROM MECHANISM TO PHENOMENON IS NOT INJECTIVE
0:05:48 GOLDENFELD AND KADANOFF*
0:06:47 REDUCTIONISM AND EMERGENCE
0:08:25 PARAMETER IDENTIFIABILITY AND SLOPPY MODELS
0:09:01 STRUCTURAL IDENTIFIABILITY
0:17:00 PRACTICAL IDENTIFIABILITY
0:18:32 PARAMETER ESTIMATION
0:19:30 EXAMPLE: LEAST SQUARES REGRESSION
0:20:20 MAXIMUM LIKELIHOOD ESTIMATION
0:21:23 CONFIDENCE/CREDIBLE REGIONS
0:23:03 SCORE
0:24:20 FISHER INFORMATION
0:25:10 FIM AND LEAST SQUARES
0:28:33 FIM AND CRAMER-RAO BOUND
0:30:39 FIM AND STRUCTURAL IDENTIFIABILITY
0:31:41 FIM AND PRACTICAL IDENTIFIABILITY
0:32:38 SLOPPINESS
0:36:39 SLOPPINESS AND THE FIM
0:39:00 DEFINING SLOPPINESS?
0:42:13 INFORMATION GEOEMTRY
0:42:17 DEFINITIONS
0:42:59 FITTING POLYNOMIALS
0:45:20 PARAMETERIZATION DEPENDENCE
0:47:45 INFORMATION GEOMETRY
0:48:37 TWO EXPONENTIAL EXAMPLE:
0:49:14 DATA SPACE:
0:52:05 REVIEW OF IMPORTANT GEOMETRIC CONCEPTS
0:52:30 EMBEDDING SPACE
0:57:35 LEAST SQUARES EMBEDDING
0:58:55 RELATION BETWEEN EMBEDDINGS*
1:00:00 INTRINSIC VS. EXTRINSIC
1:03:05 VISUALIZATIONS
1:04:33 GALLERY OF MODEL MANIFOLDS
1:06:37 GEODESICS
1:07:02 CURVATURE
1:07:06 GEOMETRIC SLOPPINESS: WIDTHS AND CURVATURES
1:09:57 INTERPOLATION (PREVIEW)
1:10:07 EXTENDED GEODESIC COORDINATES
1:10:25 OPTIMAL EXPERIMENTAL DESIGN
1:10:31 PROBLEM STATEMENT
1:11:07 COMPLEMENTARY EXPERIMENTS
1:12:12 OED GENERAL STRATEGY (D-OPTIMAL)
1:12:59 PREDICTIONS VS. PARAMETERS
1:13:45 SLOPPINESS AND THE ROLE OF EXPERIMENTAL DESIGN
1:14:37 ESTIMATING PARAMETERS OF BROWN ET AL.
1:15:13 HOW MUCH DATA IS NECESSARY?
1:16:35 THE CAUSE AND CURE OF SLOPPINESS
1:17:31 THE LIMITATIONS OF OED
1:19:32 DNA REPAIR
1:21:49 MODELING MODEL ERROR
1:22:58 EGFR SIGNALING REVISITED
1:24:22 PARAMETERS WITHOUT PREDICTIONS
1:24:57 UNCERTAINTY QUANTIFICATION
1:25:25 FUNDAMENTAL LIMITS TO PARAMETER ESTIMATION
1:26:19 ESTIMATING MODEL ERROR IN SLOPPY SYSTEM
1:27:51 REDUCTIONISM, MODELING, AND OED
1:28:36 RELEVANT VS. IRRELEVANT PARAMETERS
1:29:33 Q&A

More from this channel