\documentclass{article} \usepackage[margin=1in]{geometry} \usepackage{amsmath,amsthm,amssymb} \usepackage{relsize} \newcounter{lecnum} \usepackage{graphicx} \graphicspath{./} \usepackage{caption} \usepackage{subcaption} \newcommand{\abs}[1]{\lvert #1 \rvert} \newcommand{\lecture}[4]{ \newpage \setcounter{lecnum}{#1} \noindent \begin{center} \framebox{ \vbox{\vspace{2mm} \hbox to 16cm { {\bf CS602 Applied Algorithms \hfill 2019-20 Sem II} } \vspace{4mm} \hbox to 16cm { {\Large \hfill Lecture #1: #2 \hfill} } \vspace{2mm} \hbox to 16cm { {\it Scribe: #4 \hfill Lecturer: #3} } \vspace{2mm}} } \end{center} \vspace*{4mm} } \newtheorem{theorem}{Theorem}[lecnum] \newtheorem{lemma}[theorem]{Lemma} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{claim}[theorem]{Claim} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{definition}[theorem]{Definition} \begin{document} \lecture{10}{February 6}{Rohit Gurjar}{Debtanu Pal} \section{Maximum Matching Problem as a Linear Program} \begin{definition}{(Maximum Matching Problem)} Given a graph $G(V,E)$, a matching $M \subseteq E$ is subset of edges such that every vertex has at most one edge incident from M. The objective of the maximum matching Problem is to find a \textbf{set of matching edges} with \textbf{maximum cardinality}. \end{definition} \subsection{Integer Programming Formulation of the Matching Problem} We can write an integer programming formulation of the maximum matching Problem. % as an Optimization Problem. Consider a variable $X_e$ for each edge $e \in E$ of the graph $G(V,E)$. Any matching $M \subseteq E$ can be represented by a point $X \in \mathbb{R}^E$ such that %we define a variable $X_e$ as $X_e = 1$ if the edge $e \in M$; $X_e = 0$ otherwise. % To restrict the solution set to only matching points, we will be adding one constraint for each of the vertices of the graph $G(V,E)$. These constraints will make sure that there is only one edge is incident to each of the vertices i.e., the set of edges obtained is indeed a matching. We can write the above conditions as an integer program as : \begin{align*} \max &\sum_{e \in E} X_e \\ \forall e \in E &X_e \in \{0,1\} \\ \forall v \in V & \sum_{e \in \delta(v)} X_e \leq 1. \end{align*} where, $\delta(v) $ is the set of edges incident on the vertex $v \in V$. But as we know, integer programming is NP-hard, we need to find a better way (convert this into a linear program perhaps) to solve the maximum matching Problem. \subsection{LP Relaxation} We can relax the conditions of the above integer program to obtain a linear program as follows. We convert the condition : $X_e \in \{0,1\}$ to $0 \leq X_e \leq 1$, and hence obtain the linear program : \begin{align*} \max &\sum_{e \in E} X_e \\ \forall e \in E &0 \leq X_e \leq 1 \\ \forall v \in V & \sum_{e \in \delta(v)} X_e \leq 1. \end{align*} But, it is not necessary that the solution obtained by the LP is same as the solution to the maximum matching problem i.e., the integer program defined above. Consider for example a complete graph with 3 vertices: \begin{figure} \centering \begin{subfigure}{.5\textwidth} \centering \includegraphics[width=5cm, height=5cm]{p1.png} \caption{Maximum matching integer program solution} \label{fig:IP} \end{subfigure}% \begin{subfigure}{.5\textwidth} \centering \includegraphics[width=5cm, height=5cm]{p2.png} \caption{Solution from the relaxed linear program} \label{fig:LP} \end{subfigure} \caption{Difference between integer program and the relaxed LP} \label{diff-1} \end{figure} The maximum matching for this graph will be of size 1 and hence the integer program will give a solution shown in the Figure \ref{fig:IP}, where the optimal value is `1'. But, When we solve the relaxed linear program, we do not get this solution. Rather, we get a solution as shown in Figure \ref{fig:LP}, where the optimal value is `$\frac{3}{2}$'. Here, each edge has a fractional value in Figure \ref{fig:LP}, which does not really give us a matching. \subsection{LP Polytope} It is to be noted that the convex hull of the matching points gives us a polytope -- called matching polytope (see figure \ref{fig:LP-Pol}). Note that as the matching points are feasible for the relaxed linear program, those points are contained inside the LP polytope. And hence, the matching polytope is contained inside the LP polytope. % Since, the integer program is more constrained than the linear program, % we get a polytope which is contained inside the LP Polytope. % Also, as the matching Points are feasible for the Relaxed Linear Program, those points are contained inside the LP Polytope. %\medskip \begin{figure} \centering \includegraphics[width=9cm, height=5cm]{p4.png} \caption{Convex hull of matching points} \label{fig:LP-Pol} \end{figure} Now, as we already noted, integer programming is NP-hard an so we cannot solve it. The LP relaxation does not guarantee us an integer solution always, hence we cannot necessarily get a matching out of it. But there are some special cases (special kinds of graphs) where the linear program is guaranteed to give an integer solution (or the fractional solution can be converted to an integer solution without reducing the value of the objective function). One such case is that of \textbf{bipartite graphs}. \section{Maximum Matching as LP for Bipartite Graphs} The relaxed linear program that we obtained was : \begin{align*} \max &\sum_{e \in E} X_e \\ \forall e \in E &0 \leq X_e \leq 1 \\ \forall v \in V & \sum_{e \in \delta(v)} X_e \leq 1. \end{align*} \begin{claim} \label{c1} The relaxed LP is correct (i.e., gives an integer solution or the solution can be converted to an integral solution) for {bipartite graphs}. \end{claim} \begin{claim} \label{c2} The relaxed LP is correct when there are {no odd length cycles} in the Graph. \end{claim} The claims \ref{c1} and \ref{c2} are equivalent. This is obvious because we know that {a graph is bipartite, if and only if it has no odd length cycles}. So, if the LP is correct for either bipartite graphs or for graphs with odd length cycles, it must be correct for the other also. \begin{proposition} \label{prop-1} Given any \textbf{fractional feasible solution} for the relaxed linear program for the maximum matching in a bipartite graph, we can \textbf{convert it into an integral feasible solution} for which the value of the objective function is greater than or equal to the value of the objective function for the fractional solution, i.e., \textbf{integral optimal} $=$ \textbf{fractional optimal}. \end{proposition} \begin{proof} We will describe a procedure for converting any fractional solution to an integral solution, while maintaining feasibility. For a given solution, let us call an edge fractional, if the corresponding edge variable takes a fractional value $(0 < X_e < 1)$ in the solution. We assume that there is at least one fractional edge. We can divide our procedure in two cases:- %Our objective is to convert all the edge variables with fractional value to integral values. \begin{enumerate} \item When there are no neighbouring fractional edges for a fractional edge: Let us say we have an edge $e \in E$, which has some non-zero fractional value $\alpha$. Note that, if none of its neighbouring edge variables has a fractional value, then all it's neighbours must have value \textbf{0}. Because, if any of them is \textbf{1}, then it will violate the constraint ``$\sum_{e \in \delta(v)} X_e \leq 1$" for one of end points $v$ of the edge $e$. In this case, we can just increment the value of this edge variable by $1-\alpha$, so that the edge value becomes equal to 1. And, since all the neighbouring edges are 0 anyway, no constraints are violated. Moreover, we will have our objective function value increased by $1-\alpha$. Refer to Figure \ref{p6} for an example. \begin{figure} \centering \includegraphics[width=12cm, height=8cm]{p6.png} \caption{Fractional to Integral value Case-1} \label{p6} \end{figure} \item When there are neighbouring fractional edges for every fractional edge. Start with an arbitrary fractional edge and start walking along neighboring fractional edges. There are two sub-cases: (a) when the walk results in a cycle and (b) when the walk results in a path. \begin{enumerate} \item Fractional Neighbours forming a cycle. We will consider even length cycles here, since if there are cycles in a bipartite graph they must be of even length. Moreover, during the procedure, we will understand why it will not work for odd length cycles. Consider an even length cycle of vertices $V' = \{v_1,v_2,\ldots\ldots,v_k,v_1\}$ with all edge variables having fractional values. Define $\epsilon$ to be the minimum amongst all values in the cycle. Without loss of generality, let's say the edge $(v_1, v_{2})$ has value $X_{(v_1, v_{2})} = \epsilon$. Subtract the value $\epsilon$ from $X_{(v_{i},v_{i+1})}$ add $\epsilon$ to $X_{(v_{i+1}, v_{i+2})}$ for all odd $i$ between $1$ and $k$ (consider $v_{k+1}= v_1$). This keeps all the constraints corresponding to the vertices $v_i$, for $1\leq i\leq k$, satisfied. See figure \ref{p8}. After each such iteration, at least one variable in the cycle is guaranteed to become integral. Continue this procedure, until all edge variables become integral or no more cycles exist in the graph. This procedure preserves correctness since, all edge variables $X_e$ are still between 0 and 1, and the constraint $ \sum_{e \in \delta(v)} X_e \leq 1$ corresponding to each vertex $v \in V$ is also satisfied after the completion of this procedure. Since at least one edge variable in the cycle becomes integral on every iteration, it will be a finite process. The objective function value will not decrease after the procedure (it will remain the same, since we added/subtracted $\epsilon$ an equal number of times to/from edge variables). Note that \textbf{this procedure would not work for odd length cycles}, because we cannot add/subtract $\epsilon$ same number of times to/from the edge variables in this case. See figure \ref{p9}. Here, before completing the procedure, it is not sure what should be done for $X_5$. If we add $\epsilon$ to $X_5$, the constraint at $v_1$ gets violated as $X_5 + X_1 $ will become $(0.3+(\epsilon=0.2)) +(0.7+(\epsilon=0.2)) = 1.4$ (considering no more edges are incident at $v_1$). If we subtract, the objective function value will have decreased since we basically did $($objective value + $(-3\epsilon) +(2\epsilon))$ = $($objective value $-\epsilon)$. So, that is not suitable as well. Hence, this procedure does not work for odd length cycles. \begin{figure} \centering \includegraphics[width=8cm, height=6cm]{p8.png} \caption{Fractional to Integral value Case-2(a)} \label{p8} \end{figure} \begin{figure} \centering \includegraphics[width=8cm, height=6cm]{p9.png} \caption{Problem with odd length cycles in Case 2(a)} \label{p9} \end{figure} \item Fractional neighbours forming a path. Consider such a path with vertex sequence $(v_1, v_2, \dots, v_k)$. The edges $(v_1,v_2)$ and $(v_{k-1},v_k)$ are the terminal edges in the sense that the only fractional edge incident on $v_1$ $(v_k)$ is the one connecting it to $v_2$ ($v_{k-1}$). Note that any integral edges adjacent to the path must have value $0$. We modify the solution as follows: for each odd $i$ between $1$ and $k$, increase the value $X_{(v_{i},v_{i+1})}$ by $\epsilon >0$ and decrease the value $X_{(v_{i+1},v_{i+2})}$ by $\epsilon$. We choose $\epsilon$ to be the minimum value $\min\{ X_{(v_{i+1},v_{i+2})} : 1\leq i\leq k \text{ is odd }\}$. This ensures that none of the values becomes negative when we subtract $\epsilon$. Moreover, due to alternate $+/- \epsilon$ on the path, all the vertex constraints remain valid. Now, we need to see the change in the objective value. One can observe that the objective value increases if $k$ is even and remains same otherwise. % Let us say we have an edge $e \in E$ such that the value of the edge variable $X_e = \alpha$, $\alpha$ is fractional and its neighbours have edge variable values as either fractional or 0. We consider a path of 4 adjacent vertices $v_x, v_y, v_z, v_w$, such that the edge $(v_y,v_z)$ has a fractional value and at least one of its neighbours is also fractional (otherwise it becomes Case-1). We now set this edge $(v_y, v_z)$ value to 0 and add the initial value of this variable to its adjacent vertices to preserve the vertex constraints. See figure \ref{p10} for example. We want to make the value of $X_5$ integral. Here, we can do 2 things: \begin{itemize} \item set $X_5 = 1$ and to preserve the constraints at $v_2$ and $v_3$ we must then set $X_4=0$ and $X_6=0$. But this reduces the value of the objective function by 0.1 and hence, should not be done. \item Set $\epsilon = X_5$ (basically the value of the edge variable you want to make integral). Then set $X_5=0$ and add the value of $\epsilon$ from its adjacent edges namely $X_4$ and $X_6$. This will preserve the constraints at $v_2$ and $v_3$. Moreover, it increases the value of the objective function by a value of $\epsilon$ (here $0.2$). \end{itemize} \begin{figure} \centering \includegraphics[width=16cm, height=7cm]{p10.png} \caption{Fractional to integarl values Case-2(b)} \label{p10} \end{figure} \end{enumerate} We repeat this procedure till no cases like 2(a) or 2(b) is left i.e., all cases are reduced to Case 1. \end{enumerate} \end{proof} \begin{proof}[\textbf{Proof of Claim \ref{c1} and Claim \ref{c2}}] From the above procedure, it is clear that solving the LP either gives an all integral solution (all edge variable values are integers) or it gives a solution which can be converted to an all integral solution with the guarantee that it is also optimum (objective function value for the all integral solution is never less than original solution to the LP). \end{proof} \section{Dual Formulation of the Relaxed Linear Program} From the relaxed LP in Section 1.2, we can remove the $X_e \leq 1$ constraint since it is redundant. This condition is already taken care of by the second constraint i.e $\forall v \in V \; \sum_{e \in \delta(v)} X_e \leq 1$. We will formulate the dual LP using this new LP as the Primal LP. \medskip The refined relaxed LP is as follows :- \begin{eqnarray*} \max & \sum_{e \in E} X_e & \\ \forall v \in V & \sum_{e \in \delta(v)} X_e \leq 1 & \\ \forall e \in E & X_e \geq 0 & \end{eqnarray*} The dual can be formulated as follows: \begin{eqnarray*} \min & \sum_{v \in V} y_v & \\ \forall e=(u,v) \in E, & (y_u + y_v)\geq 1 & \\ & y_v \geq 0 & \end{eqnarray*} According to the rules of formulating the dual here the dual LP should have one variable corresponding to each constraint of the Primal LP (one for each vertex precisely named $y_v, \forall v \in V$), and one constraint corresponding to each variable of the Primal LP (one for each edge). And, the objective function becomes a minimizing function. \subsection{Correspondence of the Dual LP with the Vertex Cover Problem} \begin{definition}[Vertex Cover Problem] Given a graph $G(V,E)$, a \textbf{vertex cover} is any set of vertices which covers (includes) all the edges in the set $E$. More formally a set $C \subseteq V $ is a vertex cover if for each edge $(u,v) \in E$, $u \in C$ or $v \in C$. The \textbf{Minimum vertex cover} problem aims at finding the \textbf{Vertex Cover with the minimum number of vertices}. \end{definition} Now, if we look at the dual LP carefully, we see that any integer solution to this LP corresponds to the \textbf{Vertex Cover Problem}. Here is how:- Let us consider that $y_i$ can take only integer values and we have the graph $G(V,E)$. This LP gives a value to each vertex $(y_i)$, such that the value is non-negative and the second constraint ($\forall e(u,v) \in E , (y_u + y_v)\geq 1$) makes sure that for each edge at least one of the vertices has the value 1. Any feasible solution to this LP will hence cover all the edges. But since the objective is to minimize the sum over all $(y_i)_{i \in V}$, the integer solution will be such that minimum number of $y_i$'s are set to 1 satisfying all the constraints. The set $ C$ of vertices $x$ for which $y_x = 1$ in an optimal integer solution, is therefore precisely a minimum vertex cover of the graph $G(V,E)$. But, one point that is to be noted is that the solution to the dual LP does not necessarily give a vertex cover. See figure \ref{p11}, a complete graph with 3 vertices. The minimum vertex cover for this graph is of size two ($\{v_1,v_2\}$), whereas solving the LP gives the objective function value as $\frac{3}{2}$ with $y_{v_1}=y_{v_2}=y_{v_3}=0.5$. Hence, %like in the case of the maximum matching Problem, a feasible solution with all integers assigned to $v_i$ is to guaranteed to give the optimal solution for general graphs. In other words : \textbf{Optimal(dual LP) $\leq$ Minimum vertex cover} for general graphs. \begin{figure} \centering \includegraphics[width=12cm, height=5cm]{p11.png} \caption{Solution for Vertex cover LP and the Actual Vertex Cover} \label{p11} \end{figure} \subsection{Proving K\"{o}nig's Theorem using LP Duality} We have the following results till now:- \medskip \begin{equation} \label{e1} \text{size(Maximum Matching)} \leq \text{Optimal(Primal LP) \{from Section 1\}} \end{equation} \begin{equation} \label{e2} \text{Optimal(Dual LP)} \leq \text{size(Vertex Cover Problem) \{from Section 3.1\}} \end{equation} \begin{equation} \label{e3} \text{Optimal(Primal LP)} \leq \text{Optimal(Dual LP) \{from Weak Duality\} } \end{equation} \medskip \begin{theorem} \textbf{size(Maximum Matching) $\leq$ size(Minimum Vertex Cover)} \end{theorem} \begin{proof} Combine the equations \ref{e1}, \ref{e2}, \ref{e3}. \end{proof} \medskip \begin{theorem}[K\"{o}nig's Theorem] In a bipartite graph, the size of the maximum matching is equal to the size of the minimum vertex cover. \end{theorem} \begin{proof} From section 2, we have that for a Bipartite graph, \begin{center} size(Maximum Matching) = Optimal(Primal LP) \end{center} Again, from the strong Duality theorem we have, \begin{center} Optimal(Primal LP) = Optimal(Dual LP) \end{center} We want to show : \begin{center} size(Maximum Matching) = size(Minimum Vertex Cover) \end{center} for bipartite graphs. Hence, we need to prove that : \begin{center} Optimal(Dual LP) = size(Minimum Vertex Cover) \end{center} for bipartite graphs. \begin{claim} Any optimal solution of the vertex cover LP (dual LP) is integral for bipartite Graphs. Hence, for bipartite graphs Optimal(Dual LP) = size(Minimum Vertex Cover). \end{claim} \begin{proof} Let $U_1, U_2$ be the two parts of the bipartition of the vertices. Let $y^*=\{y_1,y_2,......y_k\}, k=\abs{U_1 \cup U_2}$ be an optimal solution to the vertex cover LP chosen such that $y^*$ has the maximum number of integer components. It does not make much sense that $y^*$ would have any component $>1$ (the objective function will not be minimum). So assume $\forall i \in \{1,2, \dots, k\}$, $y_i \leq 1$. Let $F$ be the set of fractional vertices $F=\{v \in U_1 \cup U_2 \mid 0