0/参考网址
blog.51cto.com/u_15476879/…
0/前言
本篇文章一切数据集和代码均在我的GitHub中,
地址:
https://github.com/Microstrong0305/WeChat-zhihu-csdnblog-code/tree/master/Ensemble%20Learning/LightGBM
1/LightGBM分类和回归
LightGBM有两大类接口:LightGBM原生接口和scikit-learn接口(这一点和xgboost是相同的。)
而且LightGBM能够完成分类和回归两种任务。
2/分类任务
<1>根据LightGBM原生接口的分类
import numpy as np
import lightgbm
from sklearn import datasets # 数据集
from sklearn.model_selection import train_test_split # 区分练习集和测验集
from sklearn.metrics import roc_auc_score, accuracy_score # 衡量模型的好坏
# 加载数据
iris = datasets.load_iris()
# 区分练习集和测验集
train_x, test_x, train_y, test_y = train_test_split(iris.data, iris.target, test_size=0.3)
# 转换为Dataset数据格式
# 这一点和xgboost也是相同的,原生接口都需求转成特别的数据方式
train_data = lightgbm.Dataset(train_x, label=train_y)
validation_data = lightgbm.Dataset(test_x, label=test_y)
# 参数
params = {
'learning_rate': 0.1,
'lambda_l1': 0.1,
'lambda_l2': 0.2,
'max_depth': 4,
'objective': 'multiclass', # 方针函数,这个和xgboost中的参数不同
'num_class': 3,
}
# 模型练习
# 原生接口都是用的train(),这一点xgboost和lightgbm是相同的。
lightgbm_model = lightgbm.train(params,
train_data,
valid_sets=[validation_data]) # 边练习边看模型在验证集上的效果
# 模型猜测
pre_y = lightgbm_model.predict(test_x)
pre_y = [list(x).index(max(x)) for x in pre_y]
print(pre_y)
# 模型评价
print(accuracy_score(test_y, y_pred))
<2>根据Scikit-learn接口的分类
from lightgbm import LGBMClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.externals import joblib
# 加载数据
iris = load_iris()
data = iris.data
target = iris.target
# 区分练习数据和测验数据
X_train, X_test, y_train, y_test = train_test_split(data, target, test_size=0.2)
# 模型练习
# sklearn中的练习函数是fit()
gbm = LGBMClassifier(num_leaves=31, learning_rate=0.05, n_estimators=20)
gbm.fit(X_train, y_train, eval_set=[(X_test, y_test)], early_stopping_rounds=5)
# 模型存储
joblib.dump(gbm, 'loan_model.pkl')
# 模型加载
gbm = joblib.load('loan_model.pkl')
# 模型猜测
y_pred = gbm.predict(X_test, num_iteration=gbm.best_iteration_)
# 模型评价
print('The accuracy of prediction is:', accuracy_score(y_test, y_pred))
# 特征重要度
print('Feature importances:', list(gbm.feature_importances_))
# 网格查找,参数优化
estimator = LGBMClassifier(num_leaves=31)
param_grid = {
'learning_rate': [0.01, 0.1, 1],
'n_estimators': [20, 40]
}
gbm = GridSearchCV(estimator,
param_grid,
refit=True,
cv=3)
gbm.fit(X_train, y_train)
print('Best parameters found by grid search are:', gbm.best_params_)
3/回归任务
<1>根据LightGBM原生接口的回归
关于LightGBM解决回归问题,咱们用Kaggle竞赛中回归问题:House Prices: Advanced Regression Techniques,
地址:[https://www.kaggle.com/c/house-prices-advanced-regression-techniques](https://link.zhihu.com/?target=https%3A//www.kaggle.com/c/house-prices-advanced-regression-techniques)来进行实例解说。
该房价猜测的练习数据集中一共有81列,榜首列是Id,最后一列是label,中间79列是特征。
这79列特征中,有43列是类别型变量,33列是整数变量,3列是浮点型变量。
练习数据集中存在缺失值missing value
import pandas as pd
import lightgbm as lgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error # mas 绝对值误差均值
from sklearn.preprocessing import Imputer # 数据的预处理,一般是特征缩放和特征编码
# 1.读文件
data = pd.read_csv('./dataset/train.csv')
# 2.切分数据输入:特征 输出:猜测方针变量
y = data.SalePrice
X = data.drop(['SalePrice'], axis=1).select_dtypes(exclude=['object'])
# 3.切分练习集、测验集,切分份额7.5 : 2.5
train_X, test_X, train_y, test_y = train_test_split(X.values, y.values, test_size=0.25)
# 4.空值处理,默许办法:运用特征列的平均值进行填充
my_imputer = Imputer()
train_X = my_imputer.fit_transform(train_X)
test_X = my_imputer.transform(test_X)
# 5.转换为Dataset数据格式
lgb_train = lgb.Dataset(train_X, train_y)
lgb_eval = lgb.Dataset(test_X, test_y, reference=lgb_train)
# 6.参数
params = {
'task': 'train',
'boosting_type': 'gbdt', # 设置提升类型
'objective': 'regression', # 方针函数
'metric': {'l2', 'auc'}, # 评价函数
'num_leaves': 31, # 叶子节点数
'learning_rate': 0.05, # 学习速率
'feature_fraction': 0.9, # 建树的特征选择份额
'bagging_fraction': 0.8, # 建树的样本采样份额
'bagging_freq': 5, # k 意味着每 k 次迭代履行bagging
'verbose': 1 # <0 显现致命的, =0 显现错误 (正告), >0 显现信息
}
# 7.调用LightGBM模型,运用练习集数据进行练习(拟合)
# Add verbosity=2 to print messages while running boosting
my_model = lgb.train(params, lgb_train, num_boost_round=20, valid_sets=lgb_eval, early_stopping_rounds=5)
# 8.运用模型对测验集数据进行猜测
predictions = my_model.predict(test_X, num_iteration=my_model.best_iteration)
# 9.对模型的猜测成果进行评判(平均绝对误差)
print("Mean Absolute Error : " + str(mean_absolute_error(predictions, test_y)))
<2>根据Scikit-learn接口的回归
import pandas as pd
from sklearn.model_selection import train_test_split # 区分练习集和测验集
import lightgbm as lgb
from sklearn.metrics import mean_absolute_error
from sklearn.preprocessing import Imputer
# 1.读文件
data = pd.read_csv('./dataset/train.csv')
# 2.切分数据输入:特征 输出:猜测方针变量
y = data.SalePrice
X = data.drop(['SalePrice'], axis=1).select_dtypes(exclude=['object'])
# 3.切分练习集、测验集,切分份额7.5 : 2.5
train_X, test_X, train_y, test_y = train_test_split(X.values, y.values, test_size=0.25)
# 4.空值处理,默许办法:运用特征列的平均值进行填充
my_imputer = Imputer()
train_X = my_imputer.fit_transform(train_X)
test_X = my_imputer.transform(test_X)
# 5.调用LightGBM模型,运用练习集数据进行练习(拟合)
# Add verbosity=2 to print messages while running boosting
my_model = lgb.LGBMRegressor(objective='regression', num_leaves=31, learning_rate=0.05, n_estimators=20,
verbosity=2)
my_model.fit(train_X, train_y, verbose=False)
# 6.运用模型对测验集数据进行猜测
predictions = my_model.predict(test_X)
# 7.对模型的猜测成果进行评判(平均绝对误差)
print("Mean Absolute Error : " + str(mean_absolute_error(predictions, test_y)))
4/LightGBM调参
在上一部分中,LightGBM模型的参数有一部分进行了简单的设置,但大都运用了模型的默许参数,但默许参数并不是最好的。要想让LightGBM体现的更好,需求对LightGBM模型进行参数微调。下图展现的是回归模型需求调理的参数,分类模型需求调理的参数与此类似。