NL Reader

Headers: mp/nl.h and mp/nl-reader.h

NL is a format for representing optimization problems in discrete or continuous variables. It is described in the technical report Writing .nl Files.

The NL format supports a wide range of problem types including but not limited to the following areas of optimization:

This section describes the C++ API of an NL reader which is

  • Reusable: the reader can be used to process NL files in different ways and not limited to a single problem representation
  • High performance: fast mmap-based reader with SAX-like API and no dynamic memory allocations in the common case
  • Easy to use: clean, modern code base and simple API
  • Complete: supports all NL constructs including extensions implemented in AMPL Solver Library
  • Reliable: extensively and continuously tested on a variety of platforms

nl-example.cc gives a few examples of how to use the NL reader.

The mp/nl.h header only contains declarations of mp::ReadNLString and mp::ReadNLFile, and can be used to read standard optimization problem objects, for example:

#include "mp/nl.h"
#include "mp/problem.h"

mp::Problem p;
ReadNLProblem("diet.nl", p);

If you want to provide a custom NL handler, include mp/nl-reader.h instead. Note that mp/nl.h is a much smaller header than mp/nl-reader.h so prefer it unless you need access to the full NL reader API.

template <typename Handler>
void mp::ReadNLFile(fmt::CStringRef filename, Handler &handler, int flags = 0)

Reads an optimization problem in the NL format from the file filename and sends notifications of the problem components to the handler object. The handler class can be one of the following

The filename argument can be a C string or an std::string object. flags can be either 0, which is the default, to read all constructs in the order they appear in the input, or mp::READ_BOUNDS_FIRST to read variable bounds after the NL header and before other constructs such as nonlinear expressions.

Example:

// Count the number of variable references in all nonlinear expressions.
struct VarCounter : mp::NullNLHandler<int> {
  int num_vars;
  VarCounter() : num_vars(0) {}
  Reference OnVariableRef(int) {
    ++num_vars;
    return Reference();
  }
};
VarCounter counter;
mp::ReadNLFile("test.nl", counter);
fmt::print("The number of variable references is {}.", counter.num_vars);

template <typename Handler>
void mp::ReadNLString(NLStringRef str, Handler &handler, fmt::CStringRef name = "(input)", int flags = 0)

Reads an optimization problem in the NL format from the string str and sends notifications of the problem components to the handler object. The handler class can be one of the following

Both str and name can be C strings or std::string objects. The name argument is used as the name of the input when reporting errors. flags can be either 0, which is the default, to read all constructs in the order they appear in the input, or mp::READ_BOUNDS_FIRST to read variable bounds after the NL header and before other constructs such as nonlinear expressions.

class mp::NLStringRef

A reference to a null-terminated string with size.

Public Functions

NLStringRef(const char *s)

Constructs a string reference object from a C string computing the size with std::strlen.

NLStringRef(const char *s, std::size_t size)

Constructs a string reference object from a C string and a size.

NLStringRef(const std::string &s)

Constructs a string reference from an std::string object.

const char *c_str() const

Returns the pointer to a C string.

std::size_t size() const

Returns the string size.

template <typename Impl, typename ExprType>
class mp::NLHandler

An NL handler.

NLHandler can be used as a base class for other handlers. Subclasses only need to redefine methods that handle constructs they are interested in and, possibly, the types used by these methods.

Impl is a type derived from NLHandler that will receive notifications of unhandled constructs via OnUnhandled.

ExprType is a return type of expression handler methods such as OnUnary useful for building expression objects. If not used it can be any default-constructible type.

Public Types

typedef ExprType Expr

An expression type.

typedef Expr NumericExpr

A numeric expression type. It is a typedef of Expr but subclasses may define it as a different type convertible to Expr.

typedef Expr LogicalExpr

A logical expression type. It is a typedef of Expr but subclasses may define it as a different type convertible to Expr.

typedef Expr CountExpr

A count expression type. It is a typedef of Expr but subclasses may define it as a different type convertible to NumericExpr.

typedef Expr Reference

A reference expression type. It is a typedef of Expr but subclasses may define it as a different type convertible to NumericExpr.

typedef LinearExprHandler LinearObjHandler

A typedef of a class that receives notifications of terms in the linear part of an objective expression.

typedef LinearExprHandler LinearConHandler

A typedef of a class that receives notifications of terms in the linear part of a constraint expression.

typedef ArgHandler NumericArgHandler

A typedef of a class that receives notifications of numeric arguments.

typedef ArgHandler VarArgHandler

A typedef of a class that receives notifications of vararg expression arguments.

typedef ArgHandler CallArgHandler

A typedef of a class that receives notifications of call expression arguments.

typedef ArgHandler NumberOfArgHandler

A typedef of a class that receives notifications of numberof expression arguments.

typedef ArgHandler CountArgHandler

A typedef of a class that receives notifications of count expression arguments.

typedef ArgHandler LogicalArgHandler

A typedef of a class that receives notifications of logical arguments.

typedef ArgHandler PairwiseArgHandler

A typedef of a class that receives notifications of pairwise expression arguments.

typedef ArgHandler SymbolicArgHandler

A typedef of a class that receives notifications of symbolic (numeric or string) arguments.

Public Functions

virtual ~NLHandler()

Destroys the object.

void OnUnhandled(const char *kind)

Receives notification of an unhandled construct of the given kind.

Throws UnsupportedError.

void OnHeader(const NLHeader &h)

Receives notification of an NL header.

bool NeedObj(int obj_index) const

Returns true if the objective with index obj_index should be handled. This method is deprecated.

void OnObj(int index, obj::Type type, NumericExpr expr)

Receives notification of an objective type and the nonlinear part of an objective expression.

void OnAlgebraicCon(int index, NumericExpr expr)

Receives notification of the nonlinear part of an algebraic constraint expression.

void OnLogicalCon(int index, LogicalExpr expr)

Receives notification of a logical constraint expression.

LinearExprHandler BeginCommonExpr(int index, int num_linear_terms)

Receives notification of the beginning of a common expression (defined variable).

void EndCommonExpr(int index, NumericExpr expr, int position)

Receives notification of the end of a common expression.

void OnComplementarity(int con_index, int var_index, ComplInfo info)

Receives notification of a complementarity relation var_lb <= x <= var_ub complements con_lb <= body <= con_ub, where x is the variable at index var_index and body is the constraint body. info gives the constraint bounds.

LinearObjHandler OnLinearObjExpr(int obj_index, int num_linear_terms)

Receives notification of the linear part of an objective expression.

LinearConHandler OnLinearConExpr(int con_index, int num_linear_terms)

Receives notification of the linear part of a constraint expression.

LinearExprHandler OnLinearCommonExpr(int expr_index, int num_linear_terms)

Receives notification of the linear part of a common expression.

void OnVarBounds(int index, double lb, double ub)

Receives notification of variable bounds.

void OnConBounds(int index, double lb, double ub)

Receives notification of constraint bounds (ranges).

void OnInitialValue(int var_index, double value)

Receives notification of the initial value for a variable.

void OnInitialDualValue(int con_index, double value)

Receives notification of the initial value for a dual variable.

ColumnSizeHandler OnColumnSizes()

Receives notification of Jacobian column sizes.

void OnFunction(int index, fmt::StringRef name, int num_args, func::Type type)

Receives notification of a function. The name argument is a function name and it is not null-terminated.

IntSuffixHandler OnIntSuffix(fmt::StringRef name, suf::Kind kind, int num_values)

Receives notification of an integer suffix. The name argument is a suffix name and it is not null-terminated. kind specifies the suffix kind.

DblSuffixHandler OnDblSuffix(fmt::StringRef name, suf::Kind kind, int num_values)

Receives notification of a double suffix. The name argument is a suffix name and it is not null-terminated. kind specifies the suffix kind.

NumericExpr OnNumber(double value)

Receives notification of a number in a nonlinear expression.

Reference OnVariableRef(int var_index)

Receives notification of a variable reference.

Reference OnCommonExprRef(int expr_index)

Receives notification of a common expression (defined variable) reference.

NumericExpr OnUnary(expr::Kind kind, NumericExpr arg)

Receives notification of a unary expression.

NumericExpr OnBinary(expr::Kind kind, NumericExpr lhs, NumericExpr rhs)

Receives notification of a binary expression.

NumericExpr OnIf(LogicalExpr condition, NumericExpr then_expr, NumericExpr else_expr)

Receives notification of an if expression.

PLTermHandler BeginPLTerm(int num_breakpoints)

Receives notification of the beginning of a piecewise-linear term.

NumericExpr EndPLTerm(PLTermHandler handler, Reference arg)

Receives notification of the end of a piecewise-linear term.

arg: argument that is a variable or a common expression reference.

CallArgHandler BeginCall(int func_index, int num_args)

Receives notification of the beginning of a call expression.

NumericExpr EndCall(CallArgHandler handler)

Receives notification of the end of a call expression.

VarArgHandler BeginVarArg(expr::Kind kind, int num_args)

Receives notification of the beginning of a vararg expression.

NumericExpr EndVarArg(VarArgHandler handler)

Receives notification of the end of a vararg expression.

NumericArgHandler BeginSum(int num_args)

Receives notification of the beginning of a summation.

NumericExpr EndSum(NumericArgHandler handler)

Receives notification of the end of a summation.

CountArgHandler BeginCount(int num_args)

Receives notification of the beginning of a count expression.

CountExpr EndCount(CountArgHandler handler)

Receives notification of the end of a count expression.

NumberOfArgHandler BeginNumberOf(int num_args, NumericExpr arg0)

Receives notification of the beginning of a numberof expression.

NumericExpr EndNumberOf(NumberOfArgHandler handler)

Receives notification of the end of a numberof expression.

SymbolicArgHandler BeginSymbolicNumberOf(int num_args, Expr arg0)

Receives notification of the beginning of a symbolic numberof expression.

NumericExpr EndSymbolicNumberOf(SymbolicArgHandler handler)

Receives notification of the end of a symbolic numberof expression.

LogicalExpr OnBool(bool value)

Receives notification of a Boolean value.

LogicalExpr OnNot(LogicalExpr arg)

Receives notification of a logical not.

LogicalExpr OnBinaryLogical(expr::Kind kind, LogicalExpr lhs, LogicalExpr rhs)

Receives notification of a binary logical expression.

LogicalExpr OnRelational(expr::Kind kind, NumericExpr lhs, NumericExpr rhs)

Receives notification of a relational expression.

LogicalExpr OnLogicalCount(expr::Kind kind, NumericExpr lhs, CountExpr rhs)

Receives notification of a logical count expression.

LogicalExpr OnImplication(LogicalExpr condition, LogicalExpr then_expr, LogicalExpr else_expr)

Receives notification of an implication expression.

LogicalArgHandler BeginIteratedLogical(expr::Kind kind, int num_args)

Receives notification of the beginning of an iterated logical expression.

LogicalExpr EndIteratedLogical(LogicalArgHandler handler)

Receives notification of the end of an iterated logical expression.

PairwiseArgHandler BeginPairwise(expr::Kind kind, int num_args)

Receives notification of the beginning of a pairwise expression.

LogicalExpr EndPairwise(PairwiseArgHandler handler)

Receives notification of the end of a pairwise expression.

Expr OnString(fmt::StringRef value)

Receives notification of a string. The value argument is a string value and it is not null-terminated.

Expr OnSymbolicIf(LogicalExpr condition, Expr then_expr, Expr else_expr)

Receives notification of a symbolic if expression.

void EndInput()

Receives notification of the end of the input.

struct ArgHandler

A class (struct) that receives notifications of expression arguments. All argument handlers in mp::NLHandler are typedefs of this class, but subclasses of mp::NLHandler may define them as different classes.

Public Functions

void AddArg(Expr arg)

Receives notification of an argument.

struct ColumnSizeHandler

A class (struct) that receives notifications of Jacobian column sizes.

Public Functions

void Add(int size)

Receives notification of a Jacobian column size.

struct DblSuffixHandler

A class (struct) that receives notifications of double suffix values.

Public Functions

void SetValue(int index, double value)

Receives notification of a suffix value.

struct IntSuffixHandler

A class (struct) that receives notifications of integer suffix values.

Public Functions

void SetValue(int index, int value)

Receives notification of a suffix value.

struct LinearExprHandler

A class (struct) that receives notifications of terms in the linear part of a common expression.

Public Functions

void AddTerm(int var_index, double coef)

Receives notification of a term in the linear expression.

struct PLTermHandler

A class (struct) that receives notifications of slopes and breakpoints in a piecewise-linear term.

Public Functions

void AddSlope(double slope)

Receives notification of a slope.

void AddBreakpoint(double breakpoint)

Receives notification of a breakpoint.

template <typename ExprType>
class mp::NullNLHandler

An NL handler that ignores all input.

NullNLHandler can be used as a base class when only a subset of constructs needs to be handled. Unhandled constructs will be ignored, not reported.

ExprType is a return type of expression handler methods such as OnUnary useful for building expression objects. If not used it can be any default-constructible type.

Inherits from mp::NLHandler< NullNLHandler< ExprType >, ExprType >

Public Functions

void OnUnhandled(const char *)

Receives notification of an unhandled construct and ignores it.

struct mp::NLHeader

An NL header which contains information about problem dimensions, such as the number of variables and constraints, and the input format.

Base class: mp::ProblemInfo

Inherits from mp::ProblemInfo

Public Types

enum Format

Input/output format.

Values:

TEXT = 0

Text format.

The text format is fully portable meaning that an .nl file can be written on a machine of one architecture and then read on a machine of a different architecture.

BINARY = 1

Binary format.

The binary format is not generally portable and should normally be used on a single machine.

enum [anonymous]

Flags.

Values:

WANT_OUTPUT_SUFFIXES = 1

Flag that specifies whether to write output suffixes to a .sol file.

Public Members

Format format

Input/output format.

int num_ampl_options

The number of options reserved for AMPL use.

int ampl_options[MAX_AMPL_OPTIONS]

Values of options reserved for AMPL use.

Leave the default values if not using AMPL.

double ampl_vbtol

Extra info for writing a solution reserved for AMPL use.

Leave the default value if not using AMPL.

arith::Kind arith_kind

Floating-point arithmetic kind used with binary format to check if an .nl file is written using a compatible representation of floating-point numbers. It is not used with the text format and normally set to mp::arith::UNKNOWN there.

int flags

Flags. Can be either 0 or mp::NLHeader::WANT_OUTPUT_SUFFIXES.

class mp::ReadError

A read error with location information.

Inherits from Error

Public Functions

ReadError(fmt::CStringRef filename, int line, int column, fmt::CStringRef format_str, fmt::ArgList args)

Constructs the exception object.

~ReadError()

Destructs the exception object.

const std::string &filename() const

Returns the name of the file where error occurred.

int line() const

Returns the line number where error occurred, starting from 1.

int column() const

Returns the column number where error occurred, starting from 1.

class mp::BinaryReadError

A read error with information about offset in a binary input.

Inherits from Error

Public Functions

BinaryReadError(const std::string &filename, std::size_t offset, fmt::CStringRef message)

Constructs the exception object.

~BinaryReadError()

Destructs the exception object.

const std::string &filename() const

Returns the name of the file where error occurred.

std::size_t offset() const

Returns the offset in chars to the error location.

enum mp::arith::Kind

Floating-point arithmetic kind.

Values:

UNKNOWN = 0

Unknown floating-point arithmetic.

IEEE_LITTLE_ENDIAN = 1

Standard IEEE-754 floating point - little endian.

IEEE_BIG_ENDIAN = 2

Standard IEEE-754 floating point - big endian.

IBM = 3

IBM floating point.

VAX = 4

VAX floating point (legacy).

CRAY = 5

Cray floating point.

LAST = CRAY

Last floating point.

mp::READ_BOUNDS_FIRST = 1

Read variable bounds before anything else.

mp::MAX_AMPL_OPTIONS = 9

Maximum number of options reserved for AMPL use in NL and SOL formats.