Regularization method for machine learning

                  Regularization method(正则化方法)

Outline

Overview of Regularization

L0 regularization

L1 regularization

L2 regularization

Elastic Net regularization

L2,1 regularization

Model example

Reference

Overview of Regularization

Main goal:

1. Prevent over-fitting

2. Reduce prediction error

3. Improve generalization performance

Essence:

1. Constraints the parameters to be optimized

2. Minimize your error while regularizing your  parameters

L0 regularization

L1 regularization

L2 regularization

L1 vs. L2

Elastic Net regularization

L2,1 regularization

Reference

1. Sparsity and Some Basics of L1 Regularization

2. A note on the group lasso and a sparse group lasso

3. Hierarchical Structured Sparse Representation

4. 正态分布的前世今生

5. https://www.zhihu.com/question/20924039

6. Sparse methods for machine learning

				<script>
					(function(){
						function setArticleH(btnReadmore,posi){
							var winH = $(window).height();
							var articleBox = $("div.article_content");
							var artH = articleBox.height();
							if(artH > winH*posi){
								articleBox.css({
									'height':winH*posi+'px',
									'overflow':'hidden'
								})
								btnReadmore.click(function(){
									articleBox.removeAttr("style");
									$(this).parent().remove();
								})
							}else{
								btnReadmore.parent().remove();
							}
						}
						var btnReadmore = $("#btn-readmore");
						if(btnReadmore.length>0){
							if(currentUserName){
								setArticleH(btnReadmore,3);
							}else{
								setArticleH(btnReadmore,1.2);
							}
						}
					})()
				</script>
				</article>