Observations
– All above variations of regularizations works pretty successfully, as it is simple data with single predictor. It might current very important variations, when there many predictors and enormous data.
- Via scikit-learn Library
L1 Regularization
from sklearn import linear_modelalphas = np.logspace(-4,-1,4) # alphas / penality intensify. i.e., 10^4, 10^3, 10^2, 10^1
print("alphas: ", alphas)
# alphas: [0.0001 0.001 0.01 0.1 ]
# Lasso / L1 Regularization
lasso_cv = linear_model.LassoCV(alphas=alphas) # cross validation # , normalize=True
lasso_model = lasso_cv.match(X_train, Y_train) # Model Match
lasso_preds = lasso_model.predict(X_test) # Prediction
lasso_preds = [round(x,2) for x in lasso_preds]
df_lasso_coeffs = pd.DataFrame({"variable": X_train.columns.tolist(),
"Coeff":lasso_cv.coef_})
# Variable Significance i.e., mod(coeff) != 0, which might be very important for Model Match
df_lasso_coeffs[df_lasso_coeffs["Coeff"]!=0]
# variable Coeff
# Product_Sell 95.97024
print("lasso_cv.alpha_: ", lasso_cv.alpha_)
# lasso_cv.alpha_: 0.0001
def mse(true,pred):
'''
Calculate MSE price
'''
return np.indicate((pred-true)**2)
print("Loss: ", spherical(mse(Y_test,lasso_preds),2))
# Loss: 21452.96
L2 Regularization
alphas = np.logspace(-4,-1,4) # alphas / penality intensify. i.e., 10^4, 10^3, 10^2, 10^1
print("alphas: ", alphas)
# alphas: [0.0001 0.001 0.01 0.1 ]# Ridge / L2 Regularization
ridge_cv = linear_model.RidgeCV(alphas=alphas) # cross validation # , normalize=True
ridge_model = ridge_cv.match(X_train, Y_train) # Model Match
ridge_preds = ridge_model.predict(X_test) # Prediction
ridge_preds = [round(x,2) for x in ridge_preds]
df_ridge_coeffs = pd.DataFrame({"variable": X_train.columns.tolist(),
"Coeff":ridge_cv.coef_})
# Variable Significance i.e., mod(coeff) > 0.001, which might be very important for Model Match
df_ridge_coeffs[df_ridge_coeffs["Coeff"]>=0.001]
# variable Coeff
# Product_Sell 95.934556
print("ridge_cv.alpha_: ", ridge_cv.alpha_)
# ridge_cv.alpha_: 0.1
print("Loss: ", spherical(mse(Y_test,ridge_preds),2))
# Loss: 21327.3
Elastic Web Regularization
alphas = np.logspace(-4,-1,4) # alphas / penality intensify. i.e., 10^4, 10^3, 10^2, 10^1
print("alphas: ", alphas)
# alphas: [0.0001 0.001 0.01 0.1 ]# Elastic Web
elastic_net_cv = linear_model.ElasticNetCV(alphas=alphas) # cross validation # , normalize=True
elastic_net_model = elastic_net_cv.match(X_train, Y_train) # Model Match
elastic_net_preds = elastic_net_model.predict(X_test) # Prediction
elastic_net_preds = [round(x,2) for x in elastic_net_preds]
df_elastic_net_coeffs = pd.DataFrame({"variable": X_train.columns.tolist(),
"Coeff":elastic_net_cv.coef_})
# Variable Significance i.e., mod(coeff) > 0.001 which may be !=0, which might be very important for Elastic Model Match
df_elastic_net_coeffs[df_elastic_net_coeffs["Coeff"]>=0.001]
# variable Coeff
# Product_Sell 95.970134
print("elastic_net_cv.alpha_: ", elastic_net_cv.alpha_)
# elastic_net_cv.alpha_: 0.0001
print("Loss: ", spherical(mse(Y_test,elastic_net_preds),2))
# Loss: 21452.96
Keypoints
- Above Hyperparameters i.e., penality score will in all probability be found using cross validation by way of hyper parameter tuning i.e., grid or random search.
- All above variations of regularizations works pretty successfully, as it is simple data with single predictor. It might current very important variations, when there many predictors and enormous data.
- Principally regularization works for, Prevents Overfitting: Regularization discourages superior fashions which can match the teaching data too successfully, thus serving to the model generalize increased to new data.
- Simplifies Fashions: By penalizing large coefficients, regularization may end up in simpler fashions that are less complicated to interpret and use.
- Improves Stability: Regularization leads to additional safe fashions the place predictions are a lot much less delicate to small changes inside the teaching data.
