梯度下降

pipidi

zhujingdi1998@gmail.com

线性回归

准备工作

首先先从ex1data1中获取训练集合.
data = load('ex1data1.txt')    分别将他的两列数据
X = data(:, 1); y = data(:, 2);  m = length(y);  作为向量存入X,y引用中, 并且确定训练集的长度 

X = [ones(m, 1), data(:,1)]; % Add a column of ones to x
theta = zeros(2, 1); % initialize fitting parameters
iterations = 1500;
alpha = 0.01;

初始化theta(θ)和需要迭代的次数还有 学习的速率alpha.

代价函数

这里我写了一个函数,用来计算代价函数。 这个函数的公式如右图
下面的代码将特征集合进行向量化 并且求向量积。

function J = computeCost(X, y, theta)
%COMPUTECOST Compute cost for linear regression
%   J = COMPUTECOST(X, y, theta) computes the cost of using theta as the
%   parameter for linear regression to fit the data points in X and y

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta
%               You should set J to the cost.



a = sum((X*theta-y).^2);
J = a*(1/(2*m))

% =========================================================================

end

梯度下降

我们需要不断的进行迭代 根据上面的数值规定 我们需要迭代1500
通过不断的修改后面的导数项达到 得到最适合的theta 从而得到最小的 代价函数 这时候他的theta就是最拟合数据集的曲线。

```octave function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters) %GRADIENTDESCENT Performs gradient descent to learn theta % theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by % taking num_iters gradient steps with learning rate alpha

% Initialize some useful values m = length(y); % number of training examples J_history = zeros(num_iters, 1);

for iter = 1:num_iters

% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
%               theta. 
%
% Hint: While debugging, it can be useful to print out the values
%       of the cost function (computeCost) and gradient here.
%
new =(X')*(X*theta-y)

update=alpha*(1/m)*new;
theta = theta-update;

% ============================================================

% Save the cost J in every iteration    
J_history(iter) = computeCost(X, y, theta);

end

end ```

TODO数学证明推导

阅读量