Interior Point Methods

Interior Point Methods are a class of optimization algorithms for solving linear or nonlinear programming problems.


It finds the optimum solution by moving inside the polygon rather than moving around its surface.


History:


In 1984, Karmarkar "invented" interior-point method.
In 1985, Affine-scalling method was "invented" as an intuitive version of Karmarkar's algorithm.
In 1989, it was realized that Dikin(USSR) invented Affine-scaling(Barrier method) in 1967.


Interior point method was the first practical polynomial time algorithm for solving linear programing problems. Ellipsoid method's run time is polynomial, but in practice, the Interior Point Method and variants of Simplex Methods are much faster.


Here goes technical in summary:


1. Primal objective with log barrier function: G(μ) = cx - μ Σ ln(xj)


2. Central path algorithm: μ from infinity to 0.


3. Min with constraint Ax=b?

    ∇G(μ) perpendicular to Ax=b;
    <cj-μ/xj> is linear combination of A's rows;
    <cj-μ/xj> = yA for some y;


    Let sj = μ/xj, then
          yA + s = c  ==> yA ≤  c, this the dual constraints.


4, Duality gap: cx - yb = (yA+s)x - y (Ax)
                                     = sx
                                     = nμ

5. Conversely, if all sj xj = μ, then on central path.
    To follow Central Path, use "predictor-corrector".


6. Improvement direction? "Affine-scaling"
    From current x, s, μ ==>  x+dx, s+ds, μ+dμ
                                    ==> sj dxj + xj dsj = dμ     (1)
    Also A(x+dx) = b    ==> Adx = 0                      (2)
            yA + s = c        ==> (dy)A + ds = 0           (3)


    To solve (1)-(3), rescale "affine scaling", all xj = 1 ==> sj = μ
    The equations say
           μdx + ds = 1dμ
           Adx = 0                ==> dx  A
           (dy)A + ds = 0      ==> ds  A


    ==> project 1dμ into A and A




Note: some of the information comes from course "Advanced Algorithms" as follows:


MIT 6.854/18.415J: Advanced Algorithms (Fall 2014, David Karger)
MIT 6.854/18.415 Advanced Algorithms (Spring 2016, Ankur Moitra)

Memoization in Python

# -*- coding: utf-8 -*-
"""

File name: fib_mem.py

Created on Mon May 23 14:50:39 2016

Source: MITx: 6.00x Introduction to Computer Science and Programming

Output:  222232244629420445529739893461909967206666939096499764990979600

PEP8 Style Compliant:
In Spyder: Preferences -> Editor -> Code Introspection/Analysis,
 near the bottom right, check Style analysis (pep8).

The 2nd way to run it:
 - Comment out "@my_memoize"
 - Uncomment "fib = my_memoize(fib)"
 - Run

The 3rd way to run it:
 >> from fib_mem import fib
 >> print(fib(300))

Tested on Python 3.x
"""


##########################################
# Example 1


def my_memoize(f):
    cache = {}

    def helper(*x):  # Refer to Item 18 of "Effective Python"
        if x not in cache:
            cache[x] = f(*x)
        return cache[x]
    return helper

@my_memoize
def fib(n):
    if n <= 1:
        return n
    else:
        return fib(n-1) + fib(n-2)
# fib = memoize(fib)
print(fib(300))


##########################################
# Example 2 (stack overflow)
def functionDecorator(f):

    def new_f():

        print("Begin", f.__name__)

        foo() # using f() instead

        print("End", f.__name__)

        return new_f

@functionDecorator
def foo():
    print("inside foo()")

foo()
print(foo.__name__)

###############################################################


"Python is basically pseudo code, ..." -- Brett Slatkin, The author of "Effective Python".

Matrix Calculus

What's the partial derivatives (w.r.t. μ and Σ) of this function?

       

Yes, it's a beautiful formula: log-likelihood function of mvn distribution.

The answer can be found on page 40 of this book: The Matrix Cookbook

You will find equation (81), (57) and (61) are useful to get the partial derivatives.


The partial derivatives are used in Vibrato Monte Carlo method, which is a Path-wise/LRM hybrid method.

Note that there are a few alternative approaches to valuate financial derivatives which have non-differentiable payoff functions.

• Likelihood Ratio Method (LRM) 
• Mallianvin Calculus (Stochastic Calculus of Variations)
• "Vibrato" Monte Carlo Method

QuantLib in C++

QuantLib is an open-source C++ Library for quantitative analysis in Finance, and the QuantLib project was started by a few Quants in 2000. Now QuantLib project is Luigi Ballabio and ferninando Ametrano.

Secondly, QuantLib has been ported to other languages:

    R: RQuantLib
    Python: PyQL
    Java: JQuantLib
    Excel: QuantLibXL

QuantLib.org provides a very good API Doc, but you may still want to take a look at other sources for API documents. The following is a short list of links for QuantLib API Docs.

QuantLib SourceCodeBrowser
QuantLib Java API Docs
QuantLib API Docs generated by Doxygen(v0.3.4)
Implementing QuantLib
C++ Design Patterns and Derivatives Pricing 2e
QuantLib on YouTube

In addition, some commercial software products are also available: QRM, FinCAD, Numerix, SunGard-FastVal, Savvysoft, Quantifi, Pricing Partners Cie, Bloomberg, Intex.

http://libguides.caltech.edu/LindeFinance

Print a float or double in C++?

#include <iostream>
#include <bitset>
#include <cassert>

using namespace std;

int main(void)
{
const int n = sizeof(float)* 8; //32 bits
float f = 975.75;
unsigned int u;
assert(sizeof(f) == sizeof(u));
std::memcpy(&u, &f, sizeof(f));
std::cout << n << ": " << bitset<n>(u) << endl;
//32: 01000100011100111111000000000000

const int nd = sizeof(double)* 8;
double d = 975.75;
unsigned long long ull;
assert(sizeof(d) == sizeof(ull));
std::memcpy(&ull, &d, sizeof(d));
std::cout << nd << ": " << bitset<nd>(ull) << endl;
//64: 0100000010001110011111100000000000000000000000000000000000000000

std::system("pause");
return 0;
}

To confirm the conversion, please check out: http://www.binaryconvert.com/index.html

Built-in Smart Pointers in Modern C++

C++ is a general programming language that supports raw pointers. To use the raw pointers, we have to manage the memory carefully with new/delete, new[]/delete[], or perhaps C-style malloc/free pairs. The memory leak is always a potential risk -- imagine a runtime_error just occurred. Detecting tools like Valgrind or Garbage Collectors like Boehm GC[using mark-sweep algorithm] may be helpful to some extent, but it's still our responsibilities to make sure that the memory is properly managed and thus less time is left for the actual business needs.

Smart pointer is one answer in the language level. Actually, smart pointers were introduced in C++98. With the move semantics, they got even better in C+11.

Topics in C++ built-in smart pointers could be intricate if we dig them further deep into areas, such as GC algorithms, thread safety and exception safety. In this post, I'll compare the various smart pointers in a high level, and summarize it in a simple table. For the detailed discussion and the guidelines to use them, please refer to Chapter 4 of Scott Meyers' "Effective Modern C++", or <memory> on cplusplus.com.

	raw pointer	auto_ptr	unique_ptr	shared_ptr	weak_ptr
Language support	Always allowed	Deprecated in C++11	C++11 (replacing auto_ptr)	C++11	C++11
What are they	T* t T *t[n]	Wrapper of raw pointer	·   A smart ptr uniquely owned -- no two unique_ptr instances manage one object ·   It provides a limited GC ·   It contains a stored ptr and a stored deleter.  .  Move-only type.	·  A smart ptr shared ownership group ·   It contains a stored ptr and an owned ptr to control block. ·  Stored and owned ptrs may refer to one object. ·   Empty shared_ptr ·   Null shared_ptr	·  A smart ptr holding non-owning ref. to an object managed by shared_ptr. ·   It models the temp ownership
Use Cases	Almost never in practice	Prefer to unique_ptr	.  A ptr w/ exclusive ownership.  .  Used in Pimpl idiom	.  A ptr w/ shared ownership.	.  A shared_ptr like ptr in risk of dangling.
How to use	new/delete new[]/delete[]		up = make_unique<T>();//C++14 up = make_unique<T[]>(); up.get_deleter(); T* rp = up.get(); T* rp = up.release(); up.reset(p);//destroy & own p *up up->v1 shared_ptr<T> up{move(up)};	sp = make_shared<T>(n); sp = make_shared<T[]>(n); sp.use_count() sp.unique()? T* rp = sp.get(); // stored ptr sp.reset(); *sp sp->v1 sp1=allocate_shared<T>(alloc,10);	weak_ptr<T> wp(sp); sp1=wp.lock() wp.use_count() wp.expired()? wp.reset();
Pros			·   Small and fast. Little overhead over raw pointer. ·  Easy to convert to shared_ptr  . Allowed to custom deleter (using lambda expression)  . Capture closure support	. Low overhead (2 x unique_ptr)  . Works in multi-threaded         environments.	.  Prevent shared_ptr cycles.  .  shared_ptr <==> weak_ptr
Cons			·  Not capoyable	.  Circular reference	.  Exception: bad_weak_ptr

"The present is the past rolled up for action, and the past is the present unrolled for understanding." - Will Durant. 

REPL, Online IDE and Tools for Static Code Analysis

The code in general programming languages like Java and C++ and code is usually compiled and tested in IDEs. In other scripting languages, it's common to see a REPL (read-eval-print loop) language shell -- interactive interpreter.

REPL environment allows us to run the code piece by piece. This is very handy for testing purpose sometimes. Now there are some solutions in Java and C++.
[My thinking of picking a pair of similar tools comes from Hotelling's law -- "Linear City Model", though many more other tools are available, too.]

1. cint and igcc are two REPL simulators for C/C++.
2. javarepl and Eclipse's "scrap book" are two REPL simulators for Java.
3. ideone, codechef, and coding-ground provide online compiler suites for various programming languages(C++, Java, Scala, R, Python) by using cloud computing technologies.
4. cppcheck and cpplint.py are two tools for C++ static code analysis.

Coding ground is my favorite. It supports almost all popular languages, and claims 100% cloud. Best of all, it displays the command line and allows me to change the compiling options!

* As of December 2015, coding ground works well on my PC. It has an Android app for Tutorialspoint, but it's slow and Coding Ground on my Galaxy Note 4 is not working as well as it is on PCs.