|
| SolverDDP (boost::shared_ptr< ShootingProblem > problem) |
| Initialize the DDP solver. More...
|
|
virtual void | allocateData () |
| Allocate all the internal data needed for the solver. More...
|
|
virtual void | backwardPass () |
| Run the backward pass (Riccati sweep) More...
|
|
virtual double | calcDiff () |
| Update the Jacobian, Hessian and feasibility of the optimal control problem. More...
|
|
virtual void | computeDirection (const bool recalc=true) |
| Compute the search direction \((\delta\mathbf{x}^k,\delta\mathbf{u}^k)\) for the current guess \((\mathbf{x}^k_s,\mathbf{u}^k_s)\). More...
|
|
virtual void | computeGains (const std::size_t t) |
| Compute the feedforward and feedback terms using a Cholesky decomposition. More...
|
|
void | decreaseRegularization () |
| Decrease the state and control regularization values by a regfactor_ factor. More...
|
|
| DEPRECATED ("Use get_reg_incfactor() or get_reg_decfactor()", double get_regfactor() const ;) double get_reg_min() const |
| Return the regularization factor used to decrease / increase it. More...
|
|
| DEPRECATED ("Use get_reg_max()", double get_regmax() const) |
|
| DEPRECATED ("Use get_reg_min()", double get_regmin() const) |
|
| DEPRECATED ("Use set_reg_incfactor() or set_reg_decfactor()", void set_regfactor(const double reg_factor);) void set_reg_min(const double regmin) |
| Modify the regularization factor used to decrease / increase it. More...
|
|
| DEPRECATED ("Use set_reg_max()", void set_regmax(const double regmax)) |
|
| DEPRECATED ("Use set_reg_min()", void set_regmin(const double regmin)) |
|
virtual const Eigen::Vector2d & | expectedImprovement () |
| Return the expected improvement \(dV_{exp}\) from a given current search direction \((\delta\mathbf{x}^k,\delta\mathbf{u}^k)\). More...
|
|
virtual void | forwardPass (const double stepLength) |
| Run the forward pass or rollout. More...
|
|
const std::vector< double > & | get_alphas () const |
| Return the set of step lengths using by the line-search procedure. More...
|
|
const std::vector< MatrixXdRowMajor > & | get_K () const |
| Return the feedback gains \(\mathbf{K}_{s}\). More...
|
|
const std::vector< Eigen::VectorXd > & | get_k () const |
| Return the feedforward gains \(\mathbf{k}_{s}\). More...
|
|
const std::vector< Eigen::VectorXd > & | get_Qu () const |
| Return the Jacobian of the Hamiltonian function \(\mathbf{Q}_{\mathbf{u}_s}\). More...
|
|
const std::vector< Eigen::MatrixXd > & | get_Quu () const |
| Return the Hessian of the Hamiltonian function \(\mathbf{Q}_{\mathbf{uu}_s}\). More...
|
|
const std::vector< Eigen::VectorXd > & | get_Qx () const |
| Return the Jacobian of the Hamiltonian function \(\mathbf{Q}_{\mathbf{x}_s}\). More...
|
|
const std::vector< Eigen::MatrixXd > & | get_Qxu () const |
| Return the Hessian of the Hamiltonian function \(\mathbf{Q}_{\mathbf{xu}_s}\). More...
|
|
const std::vector< Eigen::MatrixXd > & | get_Qxx () const |
| Return the Hessian of the Hamiltonian function \(\mathbf{Q}_{\mathbf{xx}_s}\). More...
|
|
double | get_reg_decfactor () const |
| Return the regularization factor used to decrease the damping value. More...
|
|
double | get_reg_incfactor () const |
| Return the regularization factor used to increase the damping value. More...
|
|
double | get_reg_max () const |
| Return the maximum regularization value. More...
|
|
double | get_th_grad () const |
| Return the tolerance of the expected gradient used for testing the step. More...
|
|
double | get_th_stepdec () const |
| Return the step-length threshold used to decrease regularization. More...
|
|
double | get_th_stepinc () const |
| Return the step-length threshold used to increase regularization. More...
|
|
const std::vector< Eigen::VectorXd > & | get_Vx () const |
| Return the Hessian of the Value function \(V_{\mathbf{x}_s}\). More...
|
|
const std::vector< Eigen::MatrixXd > & | get_Vxx () const |
| Return the Hessian of the Value function \(V_{\mathbf{xx}_s}\). More...
|
|
void | increaseRegularization () |
| Increase the state and control regularization values by a regfactor_ factor. More...
|
|
virtual void | resizeData () |
| Resizing the solver data. More...
|
|
void | set_alphas (const std::vector< double > &alphas) |
| Modify the set of step lengths using by the line-search procedure. More...
|
|
void | set_reg_decfactor (const double reg_factor) |
| Modify the regularization factor used to decrease the damping value. More...
|
|
void | set_reg_incfactor (const double reg_factor) |
| Modify the regularization factor used to increase the damping value. More...
|
|
void | set_reg_max (const double regmax) |
| Modify the maximum regularization value. More...
|
|
void | set_th_grad (const double th_grad) |
| Modify the tolerance of the expected gradient used for testing the step. More...
|
|
void | set_th_stepdec (const double th_step) |
| Modify the step-length threshold used to decrease regularization. More...
|
|
void | set_th_stepinc (const double th_step) |
| Modify the step-length threshold used to increase regularization. More...
|
|
virtual bool | solve (const std::vector< Eigen::VectorXd > &init_xs=DEFAULT_VECTOR, const std::vector< Eigen::VectorXd > &init_us=DEFAULT_VECTOR, const std::size_t maxiter=100, const bool is_feasible=false, const double regInit=1e-9) |
| Compute the optimal trajectory \(\mathbf{x}^*_s,\mathbf{u}^*_s\) as lists of \(T+1\) and \(T\) terms. More...
|
|
virtual double | stoppingCriteria () |
| Return a positive value that quantifies the algorithm termination. More...
|
|
virtual double | tryStep (const double steplength=1) |
| Try a predefined step length \(\alpha\) and compute its cost improvement \(dV\). More...
|
|
EIGEN_MAKE_ALIGNED_OPERATOR_NEW | SolverAbstract (boost::shared_ptr< ShootingProblem > problem) |
| Initialize the solver. More...
|
|
virtual void | computeDirection (const bool recalc)=0 |
| Compute the search direction \((\delta\mathbf{x}^k,\delta\mathbf{u}^k)\) for the current guess \((\mathbf{x}^k_s,\mathbf{u}^k_s)\). More...
|
|
double | computeDynamicFeasibility () |
| Compute the dynamic feasibility \(\|\mathbf{f}_{\mathbf{s}}\|_{\infty,1}\) for the current guess \((\mathbf{x}^k,\mathbf{u}^k)\). More...
|
|
virtual const Eigen::Vector2d & | expectedImprovement ()=0 |
| Return the expected improvement \(dV_{exp}\) from a given current search direction \((\delta\mathbf{x}^k,\delta\mathbf{u}^k)\). More...
|
|
double | get_cost () const |
| Return the total cost. More...
|
|
const Eigen::Vector2d & | get_d () const |
| Return the LQ approximation of the expected improvement. More...
|
|
double | get_dV () const |
| Return the cost reduction \(dV\). More...
|
|
double | get_dVexp () const |
| Return the expected cost reduction \(dV_{exp}\). More...
|
|
double | get_ffeas () const |
| Return the feasibility of the dynamic constraints \(\|\mathbf{f}_{\mathbf{s}}\|_{\infty,1}\) of the current guess. More...
|
|
const std::vector< Eigen::VectorXd > & | get_fs () const |
| Return the gaps \(\mathbf{f}_{s}\). More...
|
|
bool | get_inffeas () const |
| Return the norm used for the computing the feasibility (true for \(\ell_\infty\), false for \(\ell_1\)) More...
|
|
bool | get_is_feasible () const |
| Return the feasibility status of the \((\mathbf{x}_s,\mathbf{u}_s)\) trajectory. More...
|
|
std::size_t | get_iter () const |
| Return the number of iterations performed by the solver. More...
|
|
const boost::shared_ptr< ShootingProblem > & | get_problem () const |
| Return the shooting problem. More...
|
|
double | get_steplength () const |
| Return the step length \(\alpha\). More...
|
|
double | get_stop () const |
| Return the value computed by stoppingCriteria() More...
|
|
double | get_th_acceptstep () const |
| Return the threshold used for accepting a step. More...
|
|
double | get_th_gaptol () const |
| Return the threshold for accepting a gap as non-zero. More...
|
|
double | get_th_stop () const |
| Return the tolerance for stopping the algorithm. More...
|
|
double | get_ureg () const |
| Return the control regularization value. More...
|
|
const std::vector< Eigen::VectorXd > & | get_us () const |
| Return the control trajectory \(\mathbf{u}_s\). More...
|
|
double | get_xreg () const |
| Return the state regularization value. More...
|
|
const std::vector< Eigen::VectorXd > & | get_xs () const |
| Return the state trajectory \(\mathbf{x}_s\). More...
|
|
const std::vector< boost::shared_ptr< CallbackAbstract > > & | getCallbacks () const |
| Return the list of callback functions using for diagnostic. More...
|
|
virtual void | resizeData () |
| Resizing the solver data. More...
|
|
void | set_inffeas (const bool inffeas) |
| Modify the current norm used for computed the feasibility. More...
|
|
void | set_th_acceptstep (const double th_acceptstep) |
| Modify the threshold used for accepting step. More...
|
|
void | set_th_gaptol (const double th_gaptol) |
| Modify the threshold for accepting a gap as non-zero. More...
|
|
void | set_th_stop (const double th_stop) |
| Modify the tolerance for stopping the algorithm. More...
|
|
void | set_ureg (const double ureg) |
| Modify the control regularization value. More...
|
|
void | set_us (const std::vector< Eigen::VectorXd > &us) |
| Modify the control trajectory \(\mathbf{u}_s\). More...
|
|
void | set_xreg (const double xreg) |
| Modify the state regularization value. More...
|
|
void | set_xs (const std::vector< Eigen::VectorXd > &xs) |
| Modify the state trajectory \(\mathbf{x}_s\). More...
|
|
void | setCallbacks (const std::vector< boost::shared_ptr< CallbackAbstract > > &callbacks) |
| Set a list of callback functions using for the solver diagnostic. More...
|
|
void | setCandidate (const std::vector< Eigen::VectorXd > &xs_warm=DEFAULT_VECTOR, const std::vector< Eigen::VectorXd > &us_warm=DEFAULT_VECTOR, const bool is_feasible=false) |
| Set the solver candidate trajectories \((\mathbf{x}_s,\mathbf{u}_s)\). More...
|
|
virtual bool | solve (const std::vector< Eigen::VectorXd > &init_xs=DEFAULT_VECTOR, const std::vector< Eigen::VectorXd > &init_us=DEFAULT_VECTOR, const std::size_t maxiter=100, const bool is_feasible=false, const double reg_init=1e-9)=0 |
| Compute the optimal trajectory \(\mathbf{x}^*_s,\mathbf{u}^*_s\) as lists of \(T+1\) and \(T\) terms. More...
|
|
virtual double | stoppingCriteria ()=0 |
| Return a positive value that quantifies the algorithm termination. More...
|
|
virtual double | tryStep (const double steplength=1)=0 |
| Try a predefined step length \(\alpha\) and compute its cost improvement \(dV\). More...
|
|
|
std::vector< double > | alphas_ |
| Set of step lengths using by the line-search procedure. More...
|
|
double | cost_try_ |
| Total cost computed by line-search procedure. More...
|
|
std::vector< Eigen::VectorXd > | dx_ |
| State error during the roll-out/forward-pass (size T) More...
|
|
Eigen::VectorXd | fTVxx_p_ |
| Store the value of \(\mathbf{\bar{f}}^T\mathbf{V_{xx}}^{'}\). More...
|
|
std::vector< MatrixXdRowMajor > | FuTVxx_p_ |
| Store the values of \(\mathbf{f_u}^T\mathbf{V_{xx}}^{'}\) per each running node. More...
|
|
MatrixXdRowMajor | FxTVxx_p_ |
| Store the value of \(\mathbf{f_x}^T\mathbf{V_{xx}}^{'}\). More...
|
|
std::vector< MatrixXdRowMajor > | K_ |
| Feedback gains \(\mathbf{K}\). More...
|
|
std::vector< Eigen::VectorXd > | k_ |
| Feed-forward terms \(\mathbf{l}\). More...
|
|
std::vector< Eigen::VectorXd > | Qu_ |
| Gradient of the Hamiltonian \(\mathbf{Q_u}\). More...
|
|
std::vector< Eigen::MatrixXd > | Quu_ |
| Hessian of the Hamiltonian \(\mathbf{Q_{uu}}\). More...
|
|
std::vector< Eigen::LLT< Eigen::MatrixXd > > | Quu_llt_ |
| Cholesky LLT solver. More...
|
|
std::vector< Eigen::VectorXd > | Quuk_ |
| Store the values of. More...
|
|
std::vector< Eigen::VectorXd > | Qx_ |
| Gradient of the Hamiltonian \(\mathbf{Q_x}\). More...
|
|
std::vector< Eigen::MatrixXd > | Qxu_ |
| Hessian of the Hamiltonian \(\mathbf{Q_{xu}}\). More...
|
|
std::vector< Eigen::MatrixXd > | Qxx_ |
| Hessian of the Hamiltonian \(\mathbf{Q_{xx}}\). More...
|
|
double | reg_decfactor_ |
| Regularization factor used to decrease the damping value. More...
|
|
double | reg_incfactor_ |
| Regularization factor used to increase the damping value. More...
|
|
double | reg_max_ |
| Maximum allowed regularization value. More...
|
|
double | reg_min_ |
| Minimum allowed regularization value. More...
|
|
double | th_grad_ |
| Tolerance of the expected gradient used for testing the step. More...
|
|
double | th_stepdec_ |
| Step-length threshold used to decrease regularization. More...
|
|
double | th_stepinc_ |
| Step-length threshold used to increase regularization. More...
|
|
std::vector< Eigen::VectorXd > | us_try_ |
| Control trajectory computed by line-search procedure. More...
|
|
std::vector< Eigen::VectorXd > | Vx_ |
| Gradient of the Value function \(\mathbf{V_x}\). More...
|
|
std::vector< Eigen::MatrixXd > | Vxx_ |
| Hessian of the Value function \(\mathbf{V_{xx}}\). More...
|
|
Eigen::MatrixXd | Vxx_tmp_ |
| Temporary variable for ensuring symmetry of Vxx. More...
|
|
Eigen::VectorXd | xnext_ |
| Next state \(\mathbf{x}^{'}\). More...
|
|
std::vector< Eigen::VectorXd > | xs_try_ |
| State trajectory computed by line-search procedure. More...
|
|
std::vector< boost::shared_ptr< CallbackAbstract > > | callbacks_ |
| Callback functions. More...
|
|
double | cost_ |
| Total cost. More...
|
|
Eigen::Vector2d | d_ |
| LQ approximation of the expected improvement. More...
|
|
double | dV_ |
| Cost reduction obtained by tryStep() More...
|
|
double | dVexp_ |
| Expected cost reduction. More...
|
|
double | ffeas_ |
| Feasibility of the dynamic constraints. More...
|
|
std::vector< Eigen::VectorXd > | fs_ |
| Gaps/defects between shooting nodes. More...
|
|
bool | inffeas_ |
|
bool | is_feasible_ |
| Label that indicates is the iteration is feasible. More...
|
|
std::size_t | iter_ |
| Number of iteration performed by the solver. More...
|
|
boost::shared_ptr< ShootingProblem > | problem_ |
| optimal control problem More...
|
|
double | steplength_ |
| Current applied step-length. More...
|
|
double | stop_ |
| Value computed by stoppingCriteria() More...
|
|
double | th_acceptstep_ |
| Threshold used for accepting step. More...
|
|
double | th_gaptol_ |
| Threshold limit to check non-zero gaps. More...
|
|
double | th_stop_ |
| Tolerance for stopping the algorithm. More...
|
|
double | tmp_feas_ |
| Temporal variables used for computed the feasibility. More...
|
|
double | ureg_ |
| Current control regularization values. More...
|
|
std::vector< Eigen::VectorXd > | us_ |
| Control trajectory. More...
|
|
bool | was_feasible_ |
| Label that indicates in the previous iterate was feasible. More...
|
|
double | xreg_ |
| Current state regularization value. More...
|
|
std::vector< Eigen::VectorXd > | xs_ |
| State trajectory. More...
|
|
Differential Dynamic Programming (DDP) solver.
The DDP solver computes an optimal trajectory and control commands by iterates running backwardPass()
and forwardPass()
. The backward-pass updates locally the quadratic approximation of the problem and computes descent direction. If the warm-start is feasible, then it computes the gaps \(\mathbf{f}_s\) and run a modified Riccati sweep:
\begin{eqnarray*}
\mathbf{Q}_{\mathbf{x}_k} &=& \mathbf{l}_{\mathbf{x}_k} + \mathbf{f}^\top_{\mathbf{x}_k} (V_{\mathbf{x}_{k+1}} +
V_{\mathbf{xx}_{k+1}}\mathbf{\bar{f}}_{k+1}),\\
\mathbf{Q}_{\mathbf{u}_k} &=& \mathbf{l}_{\mathbf{u}_k} + \mathbf{f}^\top_{\mathbf{u}_k} (V_{\mathbf{x}_{k+1}} +
V_{\mathbf{xx}_{k+1}}\mathbf{\bar{f}}_{k+1}),\\
\mathbf{Q}_{\mathbf{xx}_k} &=& \mathbf{l}_{\mathbf{xx}_k} + \mathbf{f}^\top_{\mathbf{x}_k} V_{\mathbf{xx}_{k+1}}
\mathbf{f}_{\mathbf{x}_k},\\
\mathbf{Q}_{\mathbf{xu}_k} &=& \mathbf{l}_{\mathbf{xu}_k} + \mathbf{f}^\top_{\mathbf{x}_k} V_{\mathbf{xx}_{k+1}}
\mathbf{f}_{\mathbf{u}_k},\\
\mathbf{Q}_{\mathbf{uu}_k} &=& \mathbf{l}_{\mathbf{uu}_k} + \mathbf{f}^\top_{\mathbf{u}_k} V_{\mathbf{xx}_{k+1}}
\mathbf{f}_{\mathbf{u}_k}.
\end{eqnarray*}
Then, the forward-pass rollouts this new policy by integrating the system dynamics along a tuple of optimized control commands \(\mathbf{u}^*_s\), i.e.
\begin{eqnarray}
\mathbf{\hat{x}}_0 &=& \mathbf{\tilde{x}}_0,\\
\mathbf{\hat{u}}_k &=& \mathbf{u}_k + \alpha\mathbf{k}_k + \mathbf{K}_k(\mathbf{\hat{x}}_k-\mathbf{x}_k),\\
\mathbf{\hat{x}}_{k+1} &=& \mathbf{f}_k(\mathbf{\hat{x}}_k,\mathbf{\hat{u}}_k).
\end{eqnarray}
- See also
- SolverAbstract(),
backwardPass()
and forwardPass()
Definition at line 49 of file ddp.hpp.