---
title: "SU-CS361: Derivatives, Bracketing, Descent, and Approximation Index"
source: https://www.jemoka.com/posts/kbhderivatives_descent_and_approximation/
---

Formal Formulation of Optimization constraint types of conditions FONC and SONC Derivatives Directional Derivatives numerical methods Finite-Difference Method Forward Difference Central Difference Backward Difference Complex-Difference Method exact methods: autodiff Forward Accumulation cooool: Dual Number Method Bracketing (one dimensional optimization schemes) Fibonacci Search Quadratic Search Shubert-Piyavskill Method Descent Direction Iteration Line Search Approximate Line Search Sufficient Decrease Condition Curvature Condition Trust Region Methods First-Order Methods good ol gradient descent Conjugate Gradient Hyper-gradient Descent Second-Order Methods Newton&rsquo;s Method or approximate it using Secant Method Direct Methods Cyclic Coordinate Search Accelerated Coordinate Search Powell&rsquo;s Method Hooke-Jeeves Search Generalized Pattern Search opportunistic search dynamic ordering Nelder-Mead Simplex Method
