%\VignetteIndexEntry{doptimal}
% JSS Article Template
\documentclass{article}


%%% Additional Packages %%%

% Algorithms
% http://tex.stackexchange.com/questions/229355/algorithm-algorithmic-algorithmicx-algorithm2e-algpseudocode-confused
\usepackage{algorithm}% http://ctan.org/pkg/algorithms
\usepackage{algpseudocode}% http://ctan.org/pkg/algorithmicx


% Font stuff
\usepackage[T1]{fontenc}% for correct hyphenation and T1 encoding
%\usepackage{lmodern}% latin modern font
\usepackage[american]{babel}% for American English

% Math fonts/symbols
\usepackage{amsmath}% sophisticated mathematical formulas with amstex (includes \text{})
\usepackage{mathtools}% fix amsmath deficiencies
\usepackage{amssymb}% sophisticated mathematical symbols with amstex
\usepackage{amstext}% ams mathematical fonts
\usepackage{amsfonts}% ams mathematical fonts

% Figures
\usepackage{graphicx}% for including figures

%%% Commands %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% vetors
\newcommand{\ve}[1]{\mathbf{#1}}           % for vetors
\newcommand{\sv}[1]{\boldsymbol{#1}}   % for greek letters
\newcommand{\m}[1]{\mathbf{#1}}               % for matrices
\newcommand{\sm}[1]{\boldsymbol{#1}}   % for greek letters
\newcommand{\tr}[1]{{#1}^{\mkern-1.5mu\mathsf{T}}}              % for transpose
\newcommand{\norm}[1]{||{#1}||}              % for transpose
\newcommand*{\mve}{\operatorname{ve}}
\newcommand*{\trace}{\operatorname{trace}}
\newcommand*{\rank}{\operatorname{rank}}
\newcommand*{\diag}{\operatorname{diag}}
\newcommand*{\vspan}{\operatorname{span}}
\newcommand*{\rowsp}{\operatorname{rowsp}}
\newcommand*{\colsp}{\operatorname{colsp}}
\newcommand*{\svd}{\operatorname{svd}}
\newcommand*{\edm}{\operatorname{edm}}  % euclidean distance matrix (D * D)


% statistical
\newcommand{\widebar}[1]{\overline{#1}}  

% 
% operators
\newcommand{\Had}{\circ}
\DeclareMathOperator*{\lmin}{Minimize}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\arginf}{arg\,inf}
\DeclareMathOperator*{\argsup}{arg\,sup}
%\newcommand*{\arginf}{\operatorname*{arginf}}
%\newcommand*{\argsup}{\operatorname*{argsup}}

% Sets
\newcommand*{\intersect}{\cap}
\newcommand*{\union}{\cup}
\let\oldemptyset\emptyset
\let\emptyset\varnothing

% Fields, Reals, etc. etc
\newcommand{\field}[1]{\mathbb{#1}}
\newcommand{\Reals}{\field{R}}
\newcommand{\Integers}{\field{Z}}
\newcommand{\Naturals}{\field{N}}
\newcommand{\Complex}{\field{C}}
\newcommand{\Rationals}{\field{Q}}

% Hyphenation
\hyphenation{Ar-chi-me-dean}

% Editorial
\newcommand*{\TODO}[1]{\textcolor{red}{TODO: #1}}
\newcommand*{\NOTE}[1]{\textcolor{blue}{Note: #1}}
\newtheorem{theorem}{Theorem}
\newtheorem{Proof}{Proof}

\usepackage[total={6.5in,8.75in}, top=0.9in, left=0.7in, right=0.7in, includefoot]{geometry}

% Misc
%\makeatletter
%\newcommand\myisodate{\number\year-\ifcase\month\or 01\or 02\or 03\or 04\or 05\or 06\or 07\or 08\or 09\or 10\or 11\or 12\fi-\ifcase\day\or 01\or 02\or 03\or 04\or 05\or 06\or 07\or 08\or 09\or 10\or 11\or 12\or 13\or 14\or 15\or 16\or 17\or 18\or 19\or 20\or 21\or 22\or 23\or 24\or 25\or 26\or 27\or 28\or 29\or 30\or 31\fi}% create iso date
\makeatother
%\newcommand*{\abstractnoindent}{}% define abstract such that it has no indent
%\let\abstractnoindent\abstract
%\renewcommand*{\abstract}{\let\quotation\quote\let\endquotation\endquote
%  \abstractnoindent}
%\deffootnote[1em]{1em}{1em}{\textsuperscript{\thefootnotemark}}% setting for footnote

\author{Adam Rahman}
\title{D-Optimal Experimental Design}


%% need no \usepackage{Sweave.sty}

\begin{document}
\SweaveOpts{concordance=FALSE}

\maketitle

Consider the problem of estimating a vector $\ve{x}$ from measurements $\ve{y}$ given by the relationship

\[
\ve{y} = \m{A}\ve{x} + \sv{\epsilon}, \quad \sv{\epsilon} \sim \mathcal{N}(0,1). 
\]

The variance-covariance matrix of such an estimator is proportional to $(\tr{\m{A}}\ve{A})^{-1}$. A reasonable goal during the design phase of an experiment would therefore be to minimize $(\tr{\m{A}}\ve{A})^{-1}$ in some way. 

There are many different ways in which $(\tr{\m{A}}\ve{A})^{-1}$ might be made minimal. For example, minimization of the trace of $(\tr{\m{A}}\ve{A})^{-1}$ (A-Optimality), minimization of the maximum eigenvalue of $(\tr{\m{A}}\ve{A})^{-1}$ (E-Optimality), minimization of the determinant of $(\tr{\m{A}}\ve{A})^{-1}$ (D-Optimilaity), and maximization of the trace of $\tr{\m{A}}\ve{A}$ (T-Optimality) all have their merits. 

Perhaps the most commonly used of these optimality criteria is D-Optimality, which is equivalent to maximizing the determinant of $\tr{\m{A}}\ve{A}$. Typically, the rows of $\m{A} = [\ve{a}_{1},...,\ve{a}_{q}]^{T}$ are chosen from $M$ possible test vectors $\ve{u}_{i} \in \mathcal{R}^{p}, ~ i=1,...M$, which are known in advance. That is,

\[
\ve{a}_{i} \in \{\ve{u}_{1},...,\ve{u}_{M}\}, \quad i = 1,...,q
\]

Given that the matrix $\m{A}$ is made up of these test vectors $\ve{u}_{i}$, the matrix $\tr{\m{A}}\ve{A}$ can be written as 

\begin{equation}
\tr{\m{A}}\ve{A} = q \sum_{i=1}^{M}\lambda_{i}\ve{u}_{i}\tr{\ve{u}_{i}}
\label{eq:dopt}
\end{equation}

\noindent where $\lambda_{i}$ is the fraction of rows in $\m{A}$ that are equal to the vector $\ve{u}_{i}$ \cite{vandenberghe1998determinant}.  Then, the the D-optimal experimental design problem can be written as a minimum determinant problem \cite{vandenberghe1998determinant}

\[
\begin{array}{ll}
\underset{\sv{\lambda}}{\text{minimize}} & \text{log}~\text{det}~(\sum_{i=1}^{M}\lambda_{i}\ve{u}_{i}\tr{\ve{u}_{i}})^{-1} \\
\text{subject to} & \\
& 
\begin{array}{rl}
\lambda_{i} &\geq~~0, \quad i = 1,...,m \\
\sum_{i=1}^{M}\lambda_{i} &=~~1
\end{array}
\end{array}
\]

Due to the inequality constraint, this primal formulation cannot be interpreted as a primal SQLP. By defining $\m{Z} = \ve{u}~diag(\sv{\lambda})~\tr{\ve{u}}$, the dual problem is (\cite{toh1999sdpt3}) 

\[
\begin{array}{ll}
\underset{\m{Z},~\ve{z}^{l},~\sv{\lambda}}{\text{maximize}} & \text{log det}~(\m{Z})\\
\text{subject to} &\\
& 
\begin{array}{rll}
-\sum_{i=1}^{p} \lambda_{i}(\ve{u}_{i}\tr{\ve{u}_{i}}) + \m{Z} &=~~0, & \m{Z} \in \mathcal{S}^{n} \\
-\sv{\lambda} + \ve{z}^{l} & =~~0, & \ve{z}^{l} \in \mathcal{R}^{p}_{+} \\
\ve{1}^{T}\sv{\lambda} & =~~1, & \sv{\lambda} \in \mathcal{R}^{p} \\
\end{array}
\end{array}
\]

We proceed with writing the D-Optimal design problem as an SQLP by first considering the objective function. The objective function depends only on the determinant of the matrix variable $\m{Z}$, which is the log-barrier. This indicates that the variable $v^{s}$ in the dual equation is equal to 1 in this formulation, while $v^{q}$ and $v^{l}$ are both zero. Since $\sv{\lambda}$ does not appear in the objective function, the vector $\ve{b}$ is equal to $\ve{0}$.

The constraint matrices $\m{A}$ are easy to define in the case of the dual formulation, as they multiply multiply $\sv{\lambda}$. In the first constraint, each $\lambda_{i}$ is multiplied by the matrix formed by $-\ve{u}_{i}\tr{\ve{u}}_{i}$, so define $\m{A}_{i}$ to be

\[
\m{A}_{i} = -\ve{u}_{i}\tr{\ve{u}}_{i},~~ i = 1,...,p.
\]

Then, the constraint matrix is $\m{A}^{s} = [svec(\m{A}_{1}),...,svec(\m{A}_{p})]$. In the second constraint containing the linear variable $\ve{z}^{l}$, the constraint matrix is $\m{A}^{l} = -\m{I}_{p}$, and in the third constraint containing only the unconstrained variable $\sv{\lambda}$, the constraint matrix is $\m{A}^{u} = \tr{\ve{1}}$. Since there is no quadratic variable, $A^{q} = \ve{0}$.

Finally, define the right hand side of each constraint
\[
\begin{array}{rl}
\m{C}^{s} &= \m{0}_{n\times n} \\
\m{C}^{l} &= \ve{0}_{p\times 1} \\
\m{C}^{u} &= 1
\end{array}
\]

\noindent which fully specifies the D-Optimal design problem as an SQLP. 

To convert this to a form usable by \textbf{sdpt3r}, we initialize our input variables by noting we have three blocks - $\m{X}$, $\ve{z}^{l}$, and $\sv{\lambda}$

As before, we declare the three blocks in \verb!blk!. The first block is semidefinite containing the matrix $\m{Z}$, the second a linear block containing $\ve{z}^{l}$, and the third an unrestricted block containing $\sv{\lambda}$

\begin{verbatim}
R> blk <- c("s" = n, "l" = p, "u" = 1)
\end{verbatim}

Next, by noting the variable $\sv{\lambda}$ does not appear in the objective function, we specify \verb!b! as a vector of zeros

\begin{verbatim}
R> b <- matrix(0,nrow=p,ncol=1)
\end{verbatim}

Next, looking at the right-hand side of the constraints, we define the matrices \verb!C!

\begin{verbatim}
R> C1 <- matrix(0,nrow=n,ncol=n)
R> C2 <- matrix(0,nrow=p,ncol=1)
R> C3 <- 1
R> C <- list(C1,C2,C3)
\end{verbatim}

Finally, we construct \verb!At! for each variable

\begin{verbatim}
R> A <- matrix(list(),nrow=p,ncol=1)
$>
R> for(k in 1:p){
R>   A[[k]] <- -uk %*% t(uk)
R> }
R>
R> A1 <- svec(blk[1], A,1)
R> A2 <- diag(-1,nrow=p,ncol=p)
R> A3 <- matrix(1,nrow=1,ncol=p)
R> At <- list(A1,A2,A3)
\end{verbatim}

The final hurdle necessary to address in this problem is the existence of the log-barrier. Recall that it is assumed that $v^{s}, v^{q}$, and $v^{l}$ in the dual problem are all zero in \verb!OPTIONS!. In this case, we can see that is not true, as we have a log term containing $\m{Z}$ in the objective function, meaning $v^{s}$ is equal to one. To pass this to \verb!sqlp!, we define the \verb!OPTIONS$parbarrier! variable as

\begin{verbatim}
R> control$parbarrier <- matrix(list(),nrow=3,ncol=1)
R> control$parbarrier[[1]] <- 1       #for vs
R> control$parbarrier[[2]] <- 0       #for vq
R> control$parbarrier[[3]] <- 0       #for vl
\end{verbatim}

The D-Optimal experimental design problem can now be solved using \verb!sqlp!

\begin{verbatim}
R> sqlp(blk, At, C, b, control)
\end{verbatim}

To demonstrate the output generated from a D-optimal experimental design problem, we consider a simple $3 \times 25$ matrix containing the known test vectors $\ve{u}_{1},...,\ve{u}_{25}$ (the data is available in the \verb!sqlp! package). To solve the problem using \verb!sqlp!, we use the function \verb!doptimal!, which takes as input an $n \times p$ matrix $\m{U}$ containing the known test vectors, and returns the optimal solution using \verb!sqlp!. The output we are interested in is \verb!y!, corresponding to $\sv{\lambda}$ in our formulation, the percentage of each $\ve{u}_{i}$ necessary to achieve maximum information in the experiment.

\begin{verbatim}
R> data(DoptDesign)
R> DoptDesign

    [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16]
V1 0.531 0.769 0.646 0.865 0.369 0.869 0.171 0.788 0.174 0.022 0.883 0.357 0.926 0.260 0.183 0.264
V2 0.232 0.661 0.632 0.095 0.314 0.674 0.911 0.274 0.454 0.346 0.695 0.685 0.476 0.398 0.564 0.994
V3 0.510 0.815 0.754 0.386 0.939 0.621 0.146 0.237 0.772 0.749 0.543 0.101 0.025 0.589 0.014 0.868

   [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]
V1 0.122 0.629 0.058 0.791 0.433 0.169 0.637 0.323 0.146
V2 0.310 0.482 0.663 0.218 0.018 0.097 0.254 0.407 0.153
V3 0.410 0.469 0.888 0.745 0.441 0.718 0.926 0.791 0.117

R> out <- doptimal(DoptDesign)
R> out$y

       [,1]
 [1,] 0.000
 [2,] 0.000
 [3,] 0.000
 [4,] 0.000
 [5,] 0.000
 [6,] 0.000
 [7,] 0.154
 [8,] 0.000
 [9,] 0.000
[10,] 0.000
[11,] 0.000
[12,] 0.000
[13,] 0.319
[14,] 0.000
[15,] 0.000
[16,] 0.240
[17,] 0.000
[18,] 0.000
[19,] 0.000
[20,] 0.000
[21,] 0.000
[22,] 0.000
[23,] 0.287
[24,] 0.000
[25,] 0.000
\end{verbatim}

The information matrix $\tr{\m{A}}\m{A}$ is a linear combination of the test vectors $\ve{u}_{i}$, weighted by the optimal vector \verb!y! above. 

\bibliographystyle{plain}
\bibliography{sdpt3r}
\end{document}