Basics of LINQ & Lamda Expressions

this article comes from: http://www.codeproject.com/Articles/33769/Basics-of-LINQ-Lamda-Expressions

Introduction 

Hi Friends, Its a long time since I have last written a Article. Since then, I have learned a lot of things and its time to share them with you. .NET is getting richer and richer with new concepts. The most popular amongst all of them is the introduction of Query Integration on basic Application Data.  

LINQ   

Linq is the Microsoft's first attempt to integrate queries into language. We know, it is really easy to find data from sql objects simply writing a query while its somewhat hectic when we want to do the same thing in a DataTable or Lists. Generally we will have to loop through every elements to find the exact match, if there is some aggregation we need to aggregate the values etc. Linq provides an easy way to write queries that can run with the in memory objects.  

Types of LINQ 

Linq Comes with 3 Basic Types (Provided there are lots of more types of LINQ on different type of objects :

1. LINQ (Linq to Objects)
2. DLINQ (Linq to SQL)
3. XLINQ (Linq to XML)

Before we continue with LINQ, we must look at some of the new inclusions which may come to us very handy.

Lambda Expression 

Linq is always associated with Lambda Expressions. In .NET 2.0 we have the concept of Annonymous Methods that allows you to write function body inline without the need of writing a delegate function. Lambda Expression of .NET 3.5 is to consize the concept of Annonymous function writing.
Let us take a look of how to write Annonymous methods in C# 2.0.

int i = this.compute(10);
private int compute(int value) 
{ 
   return (value + 2);
}

It is very normal example of a function call. The compute function is called when the line i=this.compute(10) is called.  We can replace the above code with inline annonymous function like this:

//C#
delegate int DelType(int i);
DelType dd =  delegate(int value)
              {
               return (value +2);
              };
int i = dd(10);

In the 2nd Case we can see that just declaring a Delegate and giving instance of that delegate type we can easily manage to write annonymous methods. .NET 3.5 puts forward this concept a little more compact. We can take the use of => operator which is introduced in .NET 3.5. It gives more flexible and easier way to write expression. Lambda Expressions can be used in the prevous case to get the result like this:

delegate int DelType(int i);
DelType d = value => value + 2;
int i = d(10);  

Thus the delegate type is assigned directly. The meaning of the line value =>value + 2 is just similar to declaring a function.  The first value indicates the arguments that we pass in a function and the one after the "=>" operator is the body of the function. Similarly if we want we can feed in as many arguments as we want to. Just in the following examples :

// Method with 2 arguments
delegate int DelType(int i, int j);
DelType d = (value1,value2) => value1 + value2;
int i = d(10,20) // Returns 30

In this example we passed in 2 arguments to the function and returns the result.  You may also wonder how can I write a lambda expression when your function body takes more than one expression. Dont worry, its simple. Take a look at the following example.

// Multiline Function Body
delegate int DelType(int i, int j);
DelType d = (value1,value2) => {
           value1 = value1 + 2;
           value2 = value2 + 2;
           return value1 + value2;
};

Thus we see writing Lambda expression is very easy. This comes very handy when working with linq, as most of the extension methods that are associated with Linq takes function delegates as arguments. We can pass simple delegate function body easily to those functions to perform work easily.

These type of expressions are also called as Expression Trees as the basic unit of this expressions are data and structured in tree format. We can also create dynamic Expression types ourself if we like to using the Expression Class.

Expression<Func<int, int>> exprTree = num => num * 5;

// Decompose the expression tree.
ParameterExpression param = (ParameterExpression)exprTree.Parameters[0];
BinaryExpression operation = (BinaryExpression)exprTree.Body;
ParameterExpression left = (ParameterExpression)operation.Left;
ConstantExpression right = (ConstantExpression)operation.Right;

Thus if we write a function which receives the ExpressionTrees we can decompose the parameters easily.

NOTE:
VB.NET doesnt support annonymous methods and lambda expressions with statement body now.

Extension Methods 

Another new concept that comes with .NET 3.5 is the Extension methods. Now we can include our own custom methods in already defined objects. We can create static classes and include custom methods to objects.  Extension method behavior is similar to that of static methods. You can declare them only in static classes. To declare an extension method, you specify the keyword this as the first parameter of the method.  Let us look at the following example:

public static class ExtensionMethods
{
   public static int ToInt32Extension(this string s)
   {
      return Int32.Parse(s);
   }
}

If we include the namespace to our application, any string variable will have ToInt32Extension method. This function takes 0 arguments and passes the string to s. We can also pass parameters just like below:

public static class ExtensionMethods
{
   public static int ToInt32ExtensionAddInteger(this string s,int value)
   {
      return Int32.Parse(s) + value;
   }
}

Here integer value is also passed. 

We can also create ExtensionMethod that takes function as argument. The class Enumerable extends a number of methods which makes use of Func class to send a method delegate within the extension so that it could work with the IEnumerable list of objects. Let us create our own extension method that can work with IEnumerable as well, just look at the code below:

public static int GetMinVal<TSource>(this IEnumerable<TSource> source,
                                    Func<TSource, int> selector)
{
     int smallest = int.MaxValue;
     foreach (var item in source)
     {
          int nextItem = selector(item);
          if (nextItem < smallest)
          {
              smallest = nextItem;
          }
     }
     return smallest;
}
// calling the Function
int minimum = employees.GetMinVal<Employee>(emp => emp.age);

Now if I call this function, it will return the Min value. The selector function will be the annonymous delegate passed in. Thus the selector will call the function like

public int selector(Employee emp)
{
    return emp.age;
}

Thus it will compare all the ages of employees, and gets the minimum of them.

Similar to this, let us write Where function ourself:

 public static IEnumerable<TSource> MyOwnWhere<TSource>(this IEnumerable<TSource> source,
                        Func<TSource, bool> predicate) where TSource : class
        {
            foreach (var item in source)
            {
                if (predicate(item))
                    yield return item;
            }
            
        }

This will call the function predicate that we pass into the function, and if the function returns true, it will grant it. Thus we make a with return of a list of source similar to Where clause as that is defined in Enumerable class as extension method.

What is Yield ? 

You may notice here I am using yield keyword to generate the list of items. This is one of the new addition to C# 2.0. The meaning of this keyword is to instruct the program to run and generate the list until all the loop execution is exhausted. Thus it creates a list of items for which the predicate function returns true.


FOR VB.NET developers 
In case of VB.NET we have to define the extension method inside a Module with Attribute
<System.Runtime.CompilerServices.Extension()>. 

Module ExtensionModule
    <System.Runtime.CompilerServices.Extension()> _
    Public Function ToInt32ExtensionAddInteger(ByVal s As String, _
                            ByVal value As Integer) As Integer
        Return Int32.Parse(s) + value
    End Function
End Module

Annonymous Type 

Annonymous types introduced with .NET can create object without even declaring the classes. The members of the annonymous types are implicitely defined during compilation

var x = new { Name ="AA", Age =30, ... };

This would create  a new object of annonymous type.

 

Meaning of VAR 

Var is a new keyword added in .NET 3.5. This is called Implicit Type Variable. During runtime it automatically assigns its type of the variable based on the object if finds.

var x = 10 // implies the variable is of integer type
var x = new list<Employee>(); // Means x  holds list of employees

var is very useful when working with annonymous typed just introduced. It assigns the type of the variable based on the type of the object passed in, so there is no difference of doing List<Employee>() x = new List<Employee>() and doing var x = new List<Employee>() as in both the case the type of the variable x is always get to List<Employee>.

Only restriction is we cannot define members of var type.

Demonstration of Linq

Let us now demonstrate LINQ with a simple example. The example can be found on the source code associated with the article.
First we define

C#
    List<Employee> employees = new List<Employee>();
    List<Employee> employee1 = new List<Employee>();
    List<Order> orders = new List<Order>();
    //Where Employee and Order are classes.
    public class Employee
    {
        public string name;
        public int age;
        public override string ToString()
        {
            return this.name;
        }
    }
    public class Order
    {
        public string itemName;
        public string empName;
        public override string ToString()
        {
            return this.itemName;
        }
    }

VB.NET

Private employees As New List(Of Employee)()
Private employee1 As New List(Of Employee)()
Private orders As New List(Of Order)()
//Where Employee and Order are classes.
Public Class Employee
    Public name As String
    Public age As Integer
    Public Overloads Overrides Function ToString() As String
        Return Me.name
    End Function
End Class
Public Class Order
    Public itemName As String
    Public empName As String
    Public Overloads Overrides Function ToString() As String
        Return Me.itemName
    End Function
End Class

We would make use of these three lists in our examples.

Operators

While writing queries, you will find a number of operators that we can use. They most general opreators are :

• Projection operators  (Select)
• Restriction operators (Where)  
• Partitioning operators (Take / Skip, TakeWhile/ SkipWhile)
• Join operators  (Join)
• Concatenation operator (Concat)
• Ordering operators  (order by)
• Grouping operators  (group by into)

Projection Operator 

With the word projection, I mean Select statements. Every linq elements should have projection in it.

C#
var iNames = from i in employees 
             select i.name;

//Using Lambda Expression
var iNames = employees.Select<Employee, string>(r => r.name);

VB.NET
Dim iNames = From i In employees _
             Select i.name

Here IEnumerable of names of employees is returned. In the above example we are also taking advantage of annonymous type declaration that was introduced in .NET 3.5.

Restriction Operator 

Restriction operator can be applied by using Where clause. Example of Where clause:

var filterEnumerable = from emp in employees
                       where emp.age > 50
                       select emp;

 //Using Lambda Expression 
var filterEnumerable = employees.Where<Employee>(emp => emp.age > 50); 

Vb.NET
Dim filterEnumerable = From emp In employees _
    Where emp.age > 50 _
    Select emp

This filters out Employees by age greater than 50.

Partitioning using Take /Skip operators 
Take can be used when we take first N elements in a list, skip will take the elements after N. 

var filterEnumerable = (from emp in employees
                         select emp).Take<Employee>(2);

 //Using Lambda Expression                                
var filterEnumerable = employees.Select<Employee, string>(r => r.name);

VB.NET
Dim filterEnumerable = (From emp In employees _
    Select emp).Take(Of Employee)(2)

Takewhile and skipwhile operator will select from a list based on a delegate passed in.

var filterEnumerable = (from emp in employees 
                        select emp).SkipWhile<Employee>(
                        r => r.name.Length > 4);

 //Using Lambda Expression 
var filterEnumerable = employees.Select<Employee, Employee>(
r => { 
       return r; 
     }).SkipWhile<Employee>(r => r.name.Length > 4); 

VB.NET
Dim filterEnumerable = (From emp In employees _
    Select emp).SkipWhile(Of Employee)(Function(r) r.name.Length > 4)

Join Operators: 

Join operators have 2 parts. The outer part gets results from inner part and vice versa so returns the result based on both

var filterEnumerable = from emp in employees
                       join ord in orders
                       on emp.name 
                       equals ord.empName 
                       select emp;

 //Using Lambda Expression 
var filterEnumerable = employees.Join<Employee, Order, string, Employee>
                       (orders, e1 => e1.name, o => o.empName, (o, e2) => o);

VB.NET 
Dim filterEnumerable = From emp In employees _
                      Join ord In orders On emp.name = ord.empName _
                      Select emp
 

Here we are joining employees and order based on the names passed in. 

Concatenation Operator : 

The Concatenation operator concats two sequence.

var filterEnumerable = (from emp in employees
                        select emp).Concat<Employee>(
                        from emp in employees1
                        select emp);

 //Using Lambda Expression 
var filterEnumerable = employees.Concat<Employee>(
                       employees1.AsEnumerable<Employee>());

VB.NET

Dim filterEnumerable = (From emp In employees _
    Select emp).Concat(Of Employee)(From emp In employees1 _
    Select emp)
 

This will concat categories of Entertainment and Food. Distinct oprator can also be used to evaluate only distinct elements in resultset. 
 

OrderBy / ThenBy     

Orderby/ThenBy can be used to order dataresults. 

var orderItems = from emp in employees 
                 orderby emp.name, emp.age descending;

  //Using Lambda Expression  
var orderItems =employees.OrderBy(i => i.name).ThenByDescending(i => i.age);

VB.NET
Dim filterEnumerable = From emp In employees Order By emp.name, emp.age  Descending

Here the ordering is done by name and then decending by age. 

GroupBy Operator :   

This is used to group elements.

var itemNamesByCategory = from i in _itemList
                          group i by i.Category into g
                          select new { Category = g.Key, Items = g };

VB.NET
Dim itemNamesByCategory = From i In _itemList _
    Group i By i.Category _
    Into g _
        Select g.Key

This gets all the categories and items grouped by category. Well this grouping seems to be a little tricky for me. Let me make you understand what exactly is the way. Here while we are grouping, we are taking the group into g(which is a IGrouping<tsource,telement>). The g will have a key, which holds the grouped data, you can add multiple grouping statement. If you want to have a having clause, its just the where clause will do. Just like the example below:

var filterEnumerable2 = from emp in employees 
                        where emp.age >65 //Normal Where clause works on all items 
                        group emp by emp.age into gr where gr.Key > 40 
                        select new { 
                                     aaa = gr.Key, 
                                     ccc=gr.Count<employee>(), 
                                     ddd=gr.Sum<employee>(r=>r.age), 
                                     bbb = gr 
                                   };

Here in the example, I have returned the aggregate functions like sum, count etc which you can find from group data.

Distinct / Union / Intersect / Except Operators : 

Union operator produces an union of two sequences

var un = (from i in _itemList
          select i.ItemName).Distinct()
          .Union((from o in _orderList
          select o.OrderName).Distinct()); 
  

Intersect operator produces an intersection of two sequences.

var inter = (from i in _itemList
             select i.ItemID).Distinct()
             .Intersect((from o in _orderList
             select o.OrderID).Distinct());

VB.NET 
Dim inter = (From i In _itemList _
    Select i.ItemID).Distinct().Intersect((From o In _orderList _
    Select o.OrderID).Distinct())

Except operator produces a set of difference elements from two sequences.

var inter = (from i in _itemList
             select i.ItemID).Distinct()
             .Except((from o in _orderList
             select o.OrderID).Distinct())

Use of let    

We can create temporary variables within LINQ statements using let keyword

var filterEnumerable = from emp in employees
                       let x = emp.age
                       let y = x * 5
                       select x;

VB.NET
Dim filterEnumerable = From emp In employees _
    Let x = emp.age _
    Let y = x * 5 _
    Select x

Here x and y are intermediate variables and we will get enumerable of integers 5 times the age of each employees.

DLINQ

DLINQ or LINQ to SQL provides a runtime framework to work with Relation Database objects. It translates LINQ queries into SQL queries automatically. We can create the total snapshot of the database using LINQ objects so that the entire relationship can be created in application classes and objects. There are tools that can create those objects automatically, we can make use of them in our program. We can make use of DataContext classes to get the flavour of DLINQ. .NET makes use of enhanced Attributes to define classes.

Let us take an example 

[Table(Name="Employee")]
  public class Employee
  {
  [Column(Id=true)]
  public string EmpID;
  
  [Column]
  public string Name;
  }

This means Employee class corresponds to the actual Employee table. Where EmpID is the primary key of the table and Name is another column.

For Further Reading:
LINQ to SQL on MSDN 

XLINQ

Similar to DLINQ, XLINQ means using LINQ to XML documents. .NET extended the little jem LINQ to create a number of classes which can manipulate XML documents very easily. Compared to XML DOM, XLINQ is much more flexible because it does not require you to always have a document object to be able to work with XML. Therefore, you can work directly with nodes and modify them as content of the document without having to start from a root XmlDocument object. This is a very powerful and flexible feature that you can use to compose larger trees and XML documents from tree fragments. Let us look at the following example:  

XDocument bookStoreXml =
      new XDocument(
       new XDeclaration("1.0", "utf-8", "yes"),
       new XComment("Bookstore XML Example"),                  
       new XElement("bookstore", new XElement("genre",
         new XAttribute("name", "Fiction"),
         new XElement("book", new XAttribute("ISBN", 
           "10-861003-324"), new XAttribute("Title", "A Tale of Two Cities"),
          new XAttribute("Price", "19.99"),                             
          new XElement("chapter", "Abstract...", new XAttribute("num", "1"),
           new XAttribute("name", "Introduction")),                        
          new XElement("chapter", "Abstract...", new XAttribute("num", "2"),
           new XAttribute("name", "Body")  
           )
          )
         )                    
        )
      );

In this example, we have created an XML document using XDocument class. The XDocument class may contain XElements which are the XML elements.XDeclaration is the class to create XML declaration. XAttribute creates an XML attributes. XComment creates XML comments. In this example Root element bookstore holds all the child elements inside it. We can  pass an entire tree of XElement inside any XElement thus it gives easier way to create XML documents.

After reading, creating or manipulating the xml document, we can save the document easily to the disk using save method on XDocument object.

bookStoreXml.Save(@"C:XMLDocumentsBookstore.xml");

It will save the entire document to the disk.

To read the document again and create the entire tree structure of object in application end we call the load method of XElement.

XElement.Load(@"C:XMLDocumentsBookstore.xml");

We can also use parse method if we want to load the xml from a string.

You can query XML document in the same way as we use in objects. We can make use of some methods Ancestors (which returns all the ancestor elements in IEnumerable list), Descendants (returns all the decendants of the element), Element(returns the child element with name), Elements(returns all the child elements of the current node), Nodes (returns the content of the current node)