SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions

About

Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at https://anonymous.4open.science/r/SOC-ICNN-4B18/.

Kang Liu, Jianchen Hu, Wei Peng · 2026

Related benchmarks

Task	Dataset	Result
Downstream Optimization	box_logsumexp d=50 (test)	Test Relative Error0.04	7
Downstream Optimization	box_socp d=50 (test)	Test Relative Error3.6	7
Learning Convex Surrogate Functions	budget_twocone_socp d=10	Test Relative Error0.035	7
Learning Convex Surrogate Functions	box_logsumexp d=10	Test Relative Error0.081	7
Learning Convex Surrogate Functions	budget_huber d=10	Test Relative Error8.9	7
Learning Convex Surrogate Functions	box_socp d=10	Test RelErr20.2	7
Learning Convex Surrogate Functions	simplex_logistic d=10	Test Relative Error17	7
Learning Convex Surrogate Functions	simplex_socp d=10	Test Relative Error12.1	7
box_logsumexp optimization	Experiment d = 20 3	Test Relative Error—	4