zoukankan      html  css  js  c++  java
  • sklearn scoring . xgboost.train . ---> rsme

    http://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

    3.3.1. The scoring parameter: defining model evaluation rules

    Model selection and evaluation using tools, such as model_selection.GridSearchCV andmodel_selection.cross_val_score, take a scoring parameter that controls what metric they apply to the estimators evaluated.

    3.3.1.1. Common cases: predefined values

    For the most common use cases, you can designate a scorer object with the scoring parameter; the table below shows all possible values. All scorer objects follow the convention that higher return values are better than lower return values. Thus metrics which measure the distance between the model and the data, like metrics.mean_squared_error, are available as neg_mean_squared_error which return the negated value of the metric.

    ScoringFunctionComment
    Classification    
    ‘accuracy’ metrics.accuracy_score  
    ‘average_precision’ metrics.average_precision_score  
    ‘f1’ metrics.f1_score for binary targets
    ‘f1_micro’ metrics.f1_score micro-averaged
    ‘f1_macro’ metrics.f1_score macro-averaged
    ‘f1_weighted’ metrics.f1_score weighted average
    ‘f1_samples’ metrics.f1_score by multilabel sample
    ‘neg_log_loss’ metrics.log_loss requires predict_proba support
    ‘precision’ etc. metrics.precision_score suffixes apply as with ‘f1’
    ‘recall’ etc. metrics.recall_score suffixes apply as with ‘f1’
    ‘roc_auc’ metrics.roc_auc_score  
    Clustering    
    ‘adjusted_rand_score’ metrics.adjusted_rand_score  
    Regression    
    ‘neg_mean_absolute_error’ metrics.mean_absolute_error  
    ‘neg_mean_squared_error’ metrics.mean_squared_error  
    ‘neg_median_absolute_error’ metrics.median_absolute_error  
    ‘r2’ metrics.r2_score  

    Usage examples:

    >>>
    >>> from sklearn import svm, datasets
    >>> from sklearn.model_selection import cross_val_score
    >>> iris = datasets.load_iris()
    >>> X, y = iris.data, iris.target
    >>> clf = svm.SVC(probability=True, random_state=0)
    >>> cross_val_score(clf, X, y, scoring='neg_log_loss') 
    array([-0.07..., -0.16..., -0.06...])
    >>> model = svm.SVC()
    >>> cross_val_score(model, X, y, scoring='wrong_choice')
    Traceback (most recent call last):
    ValueError: 'wrong_choice' is not a valid scoring value. Valid options are ['accuracy', 'adjusted_rand_score', 'average_precision', 'f1', 'f1_macro', 'f1_micro', 'f1_samples', 'f1_weighted', 'neg_log_loss', 'neg_mean_absolute_error', 'neg_mean_squared_error', 'neg_median_absolute_error', 'precision', 'precision_macro', 'precision_micro', 'precision_samples', 'precision_weighted', 'r2', 'recall', 'recall_macro', 'recall_micro', 'recall_samples', 'recall_weighted', 'roc_auc']
    

    Note

     

    The values listed by the ValueError exception correspond to the functions measuring prediction accuracy described in the following sections. The scorer objects for those functions are stored in the dictionarysklearn.metrics.SCORERS.

    3.3.1.2. Defining your scoring strategy from metric functions

    The module sklearn.metric also exposes a set of simple functions measuring a prediction error given ground truth and prediction:

    • functions ending with _score return a value to maximize, the higher the better.
    • functions ending with _error or _loss return a value to minimize, the lower the better. When converting into a scorer object using make_scorer, set the greater_is_better parameter to False (True by default; see the parameter description below).

    Metrics available for various machine learning tasks are detailed in sections below.

    Many metrics are not given names to be used as scoring values, sometimes because they require additional parameters, such as fbeta_score. In such cases, you need to generate an appropriate scoring object. The simplest way to generate a callable object for scoring is by using make_scorer. That function converts metrics into callables that can be used for model evaluation.

    One typical use case is to wrap an existing metric function from the library with non-default values for its parameters, such as the beta parameter for the fbeta_score function:

    >>>
    >>> from sklearn.metrics import fbeta_score, make_scorer
    >>> ftwo_scorer = make_scorer(fbeta_score, beta=2)
    >>> from sklearn.model_selection import GridSearchCV
    >>> from sklearn.svm import LinearSVC
    >>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]}, scoring=ftwo_scorer)
  • 相关阅读:
    简单自定义控件在view下可以运行在传统模式下运行显示空白
    symbian菜单2运行时更换菜单实现右键菜单功能
    得到文件路径信息 TParse
    Symbian菜单:创建一个最简单的菜单
    RFs 目录操作
    作品[RFileRead/WriteStream 练习] for s60 fp2
    对话框的练习
    文件浏览工具s60第二版
    转描述符
    得到驱动器信息
  • 原文地址:https://www.cnblogs.com/xinping-study/p/7233712.html
Copyright © 2011-2022 走看看