Self-assembly, the process by which interacting components form well-defined and often intricate structures, is typically thought of as a spontaneous process arising from equilibrium dynamics. When a system is driven by external nonequilibrium forces, states statistically inaccessible to the equilibrium dynamics can arise, a process sometimes termed direct self-assembly. However, if we fix a given target state and a set of external control variables, it is not well-understood (i) how to design a protocol to drive the system toward the desired state nor (ii) the cost of persistently perturbing the stationary distribution. In this work, we derive a bound that relates the proximity to the chosen target with the dissipation associated with the external drive, showing that high-dimensional external control can guide systems toward target distribution but with an inevitable cost. Remarkably, the bound holds arbitrarily far from equilibrium. Second, we investigate the performance of deep reinforcement learning algorithms and provide evidence for the realizability of complex protocols that stabilize otherwise inaccessible states of matter.

1.
Y.
Yin
and
A. P.
Alivisatos
, “
Colloidal nanocrystal synthesis and the organic–inorganic interface
,”
Nature
437
,
664
670
(
2005
).
2.
Y.
Ma
and
A. L.
Ferguson
, “
Inverse design of self-assembling colloidal crystals with omnidirectional photonic bandgaps
,”
Soft Matter
15
,
8808
8826
(
2019
).
3.
K. R.
Gadelrab
,
A. F.
Hannon
,
C. A.
Ross
, and
A.
Alexander-Katz
, “
Inverting the design path for self-assembled block copolymers
,”
Mol. Syst. Des. Eng.
2
,
539
548
(
2017
).
4.
H.
Ronellenfitsch
,
N.
Stoop
,
J.
Yu
,
A.
Forrow
, and
J.
Dunkel
, “
Inverse design of discrete mechanical metamaterials
,”
Phys. Rev. Mater.
3
,
095201
(
2019
).
5.
J. C.
Cameron
,
S. C.
Wilson
,
S. L.
Bernstein
, and
C. A.
Kerfeld
, “
Biogenesis of a bacterial organelle: The carboxysome assembly pathway
,”
Cell
155
,
1131
1140
(
2013
).
6.
C.
Sigl
,
E. M.
Willner
,
W.
Engelen
,
J. A.
Kretzmann
,
K.
Sachenbacher
,
A.
Liedl
,
F.
Kolbe
,
F.
Wilsch
,
S. A.
Aghvami
,
U.
Protzer
,
M. F.
Hagan
,
S.
Fraden
, and
H.
Dietz
, “
Programmable icosahedral shell system for virus trapping
,”
Nat. Mater.
20
,
1281
(
2021
).
7.
G. M.
Rotskoff
and
P. L.
Geissler
, “
Robust nonequilibrium pathways to microcompartment assembly
,”
Proc. Natl. Acad. Sci. U. S. A.
115
,
6341
(
2018
).
8.
E. M.
Furst
, “
Directed self-assembly
,”
Soft Matter
9
,
9039
9045
(
2013
).
9.
M. C.
Rechtsman
,
F. H.
Stillinger
, and
S.
Torquato
, “
Optimized interactions for targeted self-assembly: Application to a honeycomb lattice
,”
Phys. Rev. Lett.
95
,
228301
(
2005
).
10.
M.
Rechtsman
,
F.
Stillinger
, and
S.
Torquato
, “
Designed interaction potentials via inverse methods for self-assembly
,”
Phys. Rev. E
73
,
011406
(
2006
).
11.
M. C.
Rechtsman
,
F. H.
Stillinger
, and
S.
Torquato
, “
Self-assembly of the simple cubic lattice with an isotropic potential
,”
Phys. Rev. E
74
,
021404
(
2006
).
12.
G.-R.
Yi
,
D. J.
Pine
, and
S.
Sacanna
, “
Recent progress on patchy colloids and their self-assembly
,”
J. Phys.: Condens. Matter
25
,
193101
(
2013
).
13.
Y.
Wang
,
Y.
Wang
,
X.
Zheng
,
É.
Ducrot
,
J. S.
Yodh
,
M.
Weck
, and
D. J.
Pine
, “
Crystallization of DNA-coated colloids
,”
Nat. Commun.
6
,
7253
(
2015
).
14.
Y.
Ke
,
L. L.
Ong
,
W. M.
Shih
, and
P.
Yin
, “
Three-dimensional structures self-assembled from DNA bricks
,”
Science
338
,
1177
1183
(
2012
).
15.
D.
Chen
,
G.
Zhang
, and
S.
Torquato
, “
Inverse design of colloidal crystals via optimized patchy interactions
,”
J. Phys. Chem. B
122
,
8462
8468
(
2018
).
16.
Z.
Ma
,
E.
Lomba
, and
S.
Torquato
, “
Optimized large hyperuniform binary colloidal suspensions in two dimensions
,”
Phys. Rev. Lett.
125
,
068002
(
2020
).
17.
F.
Romano
,
J.
Russo
,
L.
Kroc
, and
P.
Šulc
, “
Designing patchy interactions to self-assemble arbitrary structures
,”
Phys. Rev. Lett.
125
,
118003
(
2020
).
18.
L. O.
Hedges
,
R. V.
Mannige
, and
S.
Whitelam
, “
Growth of equilibrium structures built from a large number of distinct component types
,”
Soft Matter
10
,
6404
6416
(
2014
).
19.
A.
Das
and
D. T.
Limmer
, “
Variational design principles for nonequilibrium colloidal assembly
,”
J. Chem. Phys.
154
,
014107
(
2021
).
20.
C. P.
Goodrich
,
E. M.
King
,
S. S.
Schoenholz
,
E. D.
Cubuk
, and
M. P.
Brenner
, “
Designing self-assembling kinetics with differentiable statistical physics models
,”
Proc. Natl. Acad. Sci. U. S. A.
118
,
e2024083118
(
2021
).
21.
R. S.
Sutton
and
A. G.
Barto
,
Reinforcement Learning: An Introduction
, 2nd ed., Adaptive Computation and Machine Learning Series (
The MIT Press
,
Cambridge, MA
,
2018
).
22.
M.
Nguyen
and
S.
Vaikuntanathan
, “
Design principles for nonequilibrium self-assembly
,”
Proc. Natl. Acad. Sci. U. S. A.
113
,
14231
14236
(
2016
).
23.

The set of functions g satisfies technical assumptions detailed in  Appendix A.

24.
C.
Villani
,
Topics in Optimal Transportation
, Graduate Studies in Mathematics Vol. 58 (
American Mathematical Society
,
Providence, RI
,
2003
).
25.
G.
Peyré
and
M.
Cuturi
, “
Computational optimal transport
,”
Found. Trends Mach. Learn.
11
,
355
(
2018
); arXiv:1803.00567 [stat].
26.
U.
Seifert
, “
Stochastic thermodynamics, fluctuation theorems, and molecular machines
,”
Rep. Prog. Phys.
75
,
126001
(
2012
); arXiv:1205.4176.
27.
S.
Vaikuntanathan
and
C.
Jarzynski
, “
Dissipation and lag in irreversible processes
,”
Europhys. Lett.
87
,
60005
(
2009
).
28.
D. A.
Sivak
and
G. E.
Crooks
, “
Near-equilibrium measurements of nonequilibrium free energy
,”
Phys. Rev. Lett.
108
,
150601
(
2012
).
29.
J. M.
Horowitz
,
K.
Zhou
, and
J. L.
England
, “
Minimum energetic cost to maintain a target nonequilibrium state
,”
Phys. Rev. E
95
,
042102
(
2017
).
30.
D. S.
Pavlichin
,
Y.
Quek
, and
T.
Weissman
, “
Minimum power to maintain a nonequilibrium distribution of a Markov chain
,” arXiv:1907.01582 [cond-mat, physics:physics] (
2019
).
31.
D. H.
Wolpert
,
A.
Kolchinsky
, and
J. A.
Owen
, “
A space–time tradeoff for implementing a function with master equation dynamics
,”
Nat. Commun.
10
,
1727
(
2019
).
32.
B.
Recht
, “
A tour of reinforcement learning: The view from continuous control
,”
Annu. Rev. Control, Robot., Auton. Syst.
2
,
253
279
(
2019
).
33.
C. J. C. H.
Watkins
and
P.
Dayan
, “
Technical note: Q-learning
,” in
Reinforcement Learning
, edited by
R. S.
Sutton
(
Springer
, Boston, MA
,
1992
), pp.
55
68
.
34.
V.
Mnih
,
K.
Kavukcuoglu
,
D.
Silver
,
A.
Graves
,
I.
Antonoglou
,
D.
Wierstra
, and
M.
Riedmiller
, “
Playing Atari with deep reinforcement learning
,” in
NIPS Deep Learning Workshop
,
2013
.
35.
S.
Fujimoto
,
H.
van Hoof
, and
D.
Meger
, “
Addressing function approximation error in actor-critic methods
,”
Proc. Mach. Learn. Res.
80
,
1587
1596
(
2018
).
36.
M. E.
Cates
and
J.
Tailleur
, “
Motility-induced phase separation
,”
Annu. Rev. Condens. Matter Phys.
6
,
219
244
(
2015
).
37.
G. S.
Redner
,
M. F.
Hagan
, and
A.
Baskaran
, “
Structure and dynamics of a phase-separating active colloidal fluid
,”
Phys. Rev. Lett.
110
,
055701
(
2013
).
38.
J.
Palacci
,
S.
Sacanna
,
A. P.
Steinberg
,
D. J.
Pine
, and
P. M.
Chaikin
, “
Living crystals of light-activated colloidal surfers
,”
Science
339
,
936
940
(
2013
).
39.
M. J.
Falk
,
V.
Alizadehyazdi
,
H.
Jaeger
, and
A.
Murugan
, “
Learning to control active matter
,”
Phys. Rev. Res.
3
,
033291
(
2021
); arXiv:2105.04641 [cond-mat].
40.

It is difficult to directly compare the duration of training because the replay buffer grows more quickly for systems with more regions of control.

41.
É.
Fodor
,
T.
Nemoto
,
S.
Vaikuntanathan
, and
L.
Tociu
, “
How dissipation constrains fluctuations in nonequilibrium liquids: Diffusion, structure, and biased interactions
,”
Phys. Rev. X
9
,
041026
(
2019
).
42.
G.
Makrides
,
B.
Zinsser
,
A.
Phinikarides
,
M.
Schubert
, and
G. E.
Georghiou
, “
Temperature and thermal annealing effects on different photovoltaic technologies
,”
Renewable Energy
43
,
407
417
(
2012
).
43.
S.
Dey
,
C.
Fan
,
K. V.
Gothelf
,
J.
Li
,
C.
Lin
,
L.
Liu
,
N.
Liu
,
M. A. D.
Nijenhuis
,
B.
Saccà
,
F. C.
Simmel
,
H.
Yan
, and
P.
Zhan
, “
DNA origami
,”
Nat. Rev. Methods Primers
1
,
13
(
2021
).
44.
A. L.
Gibbs
and
F. E.
Su
, “
On choosing and bounding probability metrics
,”
Int. Stat. Rev.
70
,
419
435
(
2002
).
You do not currently have access to this content.