Estimators

This module implements Estimator base classes.

Policies, costs, and constraints inherit from this.

class cvxportfolio.estimator.EstimatorView on GitHub

Estimator abstraction, designed for repeated evaluation over time.

Policies, costs, and constraints inherit from this. When overloading methods defined here one should be very careful. The recommended usage (if you want a class that uses our recursive execution model) is to put any sub-estimators at the class attribute level, like we do throughout the library. That ensures that the sub-estimators will be evaluated before the class itself by both initialize_estimator_recursive() and values_in_time_recursive().

initialize_estimator(universe, trading_calendar, **kwargs)View on GitHub

Initialize estimator instance with universe and trading times.

This method is called at the start of an online execution, or, in a back-test, at its start and whenever the trading universe changes. It provides the instance with the current trading universe and a pandas.DatetimeIndex representing the current and future trading calendar, i.e., the times at which the estimator will be evaluated, or a best guess of it. The instance uses these to appropriately initialize any internal object, such as Cvxpy parameters, to the right size (as implied by the universe). Also, especially for multi-period optimization and similar policies, awareness of the future trading calendar is essential to, e.g., plan in advance.

Parameters:

  • universe (pandas.Index) – Trading universe, including cash.

  • trading_calendar (pandas.DatetimeIndex) – Future (including current) trading calendar.

  • kwargs (dict) – Reserved for future expansion.

initialize_estimator_recursive(**kwargs)View on GitHub

Recursively initialize all estimators in a policy.

Parameters:

kwargs (dict) – Parameters sent down an estimator tree to inizialize it.

values_in_time(t, current_weights, current_portfolio_value, past_returns, past_volumes, current_prices, mpo_step=None, cache=None, **kwargs)View on GitHub

Evaluate estimator at current time, possibly return current value.

This method is usually the most important for Estimator classes. It is called at each point in a back-test with all data of the current state. Sub-estimators are evaluated first, in a depth-first recursive tree fashion (defined in values_in_time_recursive()). The signature differs slightly between different estimators, see below.

Parameters:

  • t (pandas.Timestamp) – Current timestamp.

  • current_weights (pandas.Series) – Current allocation weights.

  • current_portfolio_value (float) – Current total value of the portfolio in cash units.

  • past_returns (pandas.DataFrame) – Past market returns (including cash).

  • past_volumes (pandas.DataFrame or None) – Past market volumes, or None if not available.

  • current_prices (pandas.Series or None) – Current (open) prices, or None if not available.

  • mpo_step (int, optional) – For cvxportfolio.MultiPeriodOptimization which step in future planning this estimator is at: 0 is for the current step (cvxportfolio.SinglePeriodOptimization), 1 is for day ahead, …. Defaults to None if unused.

  • cache (dict, optional) – Cache or workspace shared between all elements of an estimator tree, currently only used by cvxportfolio.MultiPeriodOptimization (and derived classes). It’s useful to avoid re-computing expensive things like covariance estimates at different MPO steps. Defaults to None if unused.

  • kwargs (dict) – Reserved for future expansion.

Returns:

Current value of the estimator.

Return type:

object or None

values_in_time_recursive(**kwargs)View on GitHub

Evaluate recursively on sub-estimators.

Parameters:

kwargs (dict) – All parameters to values_in_time() that are passed to all elements contained in a policy object.

Returns:

The current value evaluated by this instance, if it implements the values_in_time() method and it returns something there.

Return type:

numpy.array, pandas.Series, pandas.DataFrame, …

finalize_estimator(**kwargs)View on GitHub

Finalize estimator instance (currently unused).

This method is called at the end of an online execution, or, in a back-test, whenever the trading universe changes (before calling initialize_estimator() with the new universe) and at its end. We aren’t currently using in the rest of the library but we plan to move the caching logic in it.

Added in version 1.1.0.

Parameters:

kwargs (dict) – Reserved for future expansion.

finalize_estimator_recursive(**kwargs)View on GitHub

Recursively finalize all estimators in a policy.

Added in version 1.1.0.

Parameters:

kwargs (dict) – Parameters sent down an estimator tree to finalize it.

class cvxportfolio.estimator.SimulatorEstimatorView on GitHub

Base class for estimators that are used by the market simulator.

Added in version 1.2.0.

This is currently used as the base class for cvxportfolio.costs.SimulatorCost but could implement in future versions more operations done as part of a simulation (back-test) loop, like filtering out trades (rejecting small ones, …), or more. It allows for nested evaluation like it’s done by values_in_time() and values_in_time_recursive(). Estimators that are used in the market simulator should derive from this. Some examples are DataEstimator, which is used to select values (like borrow costs) in simulations as well as in optimization, and cvxportfolio.forecast.HistoricalStandardDeviation, which is used also in simulation by cvxportfolio.costs.TransactionCost.

simulate(t, t_next, u, h_plus, past_volumes, past_returns, current_prices, current_returns, current_volumes, current_weights, current_portfolio_value, **kwargs)View on GitHub

Evaluate the estimator as part of a Market Simulator back-test loop.

Cost classes that are meant to be used in the simulator can implement this. The arguments to this are the same as for cvxportfolio.estimator.Estimator.values_in_time() plus the realized returns and volumes in the period, and the trades requested by the policy, ….

Parameters:

  • t (pandas.Timestamp) – Current timestamp.

  • u (pandas.Series) – Trade vector in cash units requested by the policy. If the market simulator implements rounding by number of shares and/or canceling trades on assets whose volume for the period is zero, this is after those transformations.

  • h_plus (pandas.Series) – Post-trade holdings vector.

  • past_returns (pandas.DataFrame) – Past market returns (including cash).

  • current_returns (pandas.Series) – Current period’s market returns (including cash).

  • past_volumes (pandas.DataFrame or None) – Past market volumes, or None if not available.

  • current_volumes (pandas.Series or None) – Current period’s market volumes, or None if not available.

  • current_prices (pandas.Series or None) – Current (open) prices, or None if not available.

  • current_weights (pandas.Series) – Current allocation weights (before trading).

  • current_portfolio_value (float) – Current total value of the portfolio in cash units, before costs.

  • t_next (pandas.Timestamp) – Timestamp of the next trading period.

  • kwargs (dict) – Reserved for future expansion.

Returns:

The current value of this instance.

Return type:

any object

simulate_recursive(**kwargs)View on GitHub

Evaluate simulator value(s) recursively on sub-estimators.

Parameters:

kwargs (dict) – All parameters to simulate() that are passed to all elements contained in a simulator cost object.

Returns:

The current simulator value evaluated by this instance, if it implements the simulate() method and it returns something there.

Return type:

numpy.array, pandas.Series, pandas.DataFrame, …

class cvxportfolio.estimator.CvxpyExpressionEstimatorView on GitHub

Base class for estimators that are Cvxpy expressions.

compile_to_cvxpy(w_plus, z, w_plus_minus_w_bm, **kwargs)View on GitHub

Compile term to cvxpy expression.

This is called by a Policy class on its terms before the start of the backtest to compile its Cvxpy problem. If the Policy changes in time this is called at every time step.

It can either return a scalar expression, in the case of objective terms, or a list of cvxpy constraints, in the case of constraints.

In MultiPeriodOptimization policies this is called separately for costs and constraints at different look-ahead steps with the corresponding symbolic variable objects.

Parameters:

  • w_plus (cvxpy.Variable) – Post-trade weights.

  • z (cvxpy.Variable) – Trade weights.

  • w_plus_minus_w_bm (cvxpy.Variable) – Post-trade weights minus benchmark weights.

  • kwargs (dict) – Reserved for future expansion.

class cvxportfolio.estimator.DataEstimator(data, use_last_available_time=False, allow_nans=False, compile_parameter=False, non_negative=False, positive_semi_definite=False, data_includes_cash=False, ignore_shape_check=False)View on GitHub

Estimator of point-in-time values from internal data.

It also implements logic to check that no nan are returned by its values_in_time_recursive method, which is the way Cvxportfolio objects use this class to get data, to compile and update a Cvxpy parameter, and to slice the data with the current trading universe.

Parameters:

  • data (object, pandas.Series, pandas.DataFrame) – Data expressed preferably as pandas Series or DataFrame where the first index is a pandas.DateTimeIndex. Otherwise you can pass a callable object which implements the values_in_time_recursive() method (with the standard signature) and returns the corresponding value in time, or a constant float, numpy.array, or even pandas Series or DataFrame not indexed by time (e.g., a covariance matrix where both index and columns are the stock symbols).

  • use_last_available_time (bool) – if the pandas index exists and is a pandas.DateTimeIndex you can instruct values_in_time_recursive() to retrieve the last available value at time t by setting this to True. Default is False.

  • allow_nans (bool) – If True, allow data returned to contain nan. Default False.

  • compile_parameter (bool) – If True, compile a Cvxpy parameter that gets updated with the current value of the instance at each point in a backtest. Default False.

  • non_negative (bool) – If True, the compiled Cvxpy parameter is non-negative (this affects certain Cvxpy operations). Default False.

  • positive_semi_definite (bool) – If True, the compiled Cvxpy parameter is market as a positive semi-definite matrix (this affects certain Cvxpy operations). Default False.

  • data_includes_cash (bool) – If True, when the data is sliced with the current trading universe we also look for the values corresponding to the cash account. Default False.

  • ignore_shape_check (bool) – If True, we don’t do any slicing of the data according to the current trading universe. Default False.

Raises:

  • cvxportfolio.NaNError – If nan are present in result.

  • cvxportfolio.MissingTimesError – If some times are missing.

  • cvxportfolio.MissingAssetsError – If some assets are missing.

  • cvxportfolio.DataError – If data is not in the right form.