What should you required to learn machine learning

 To learn machine learning, you will need to acquire a combination of technical skills and domain knowledge. Here are some of the things you will need to learn:

1. Mathematics and Statistics: Machine learning is based on mathematical and statistical concepts, such as linear algebra, calculus, probability, and statistics. You will need to have a strong foundation in these subjects to understand machine learning algorithms and techniques.

2. Programming: Machine learning is implemented using programming languages such as Python, R, and MATLAB. You will need to learn a programming language and be proficient in it to build machine learning models.

3. Machine learning algorithms: You will need to learn about different machine learning algorithms, including supervised learning (e.g., linear regression, decision trees, support vector machines), unsupervised learning (e.g., clustering, principal component analysis), and reinforcement learning.

4. Data preparation: Data preparation is an important step in machine learning, where you clean, transform, and preprocess the data. You will need to learn about data preparation techniques such as feature engineering, data normalization, and data cleaning.

5. Data visualization: Data visualization is an important aspect of machine learning where you can explore and understand the data. You will need to learn how to create visualizations using libraries such as Matplotlib and Seaborn.

6. Deep learning: Deep learning is a subfield of machine learning that focuses on neural networks. You will need to learn about neural networks, deep learning architectures, and frameworks such as TensorFlow, Keras, and PyTorch.

7. Domain knowledge: Domain knowledge is critical to developing machine learning solutions that are effective in solving real-world problems. You will need to learn about the specific domain you are working on and understand the context of the problem you are solving.

Overall, learning machine learning is a long-term commitment that requires a combination of technical skills, domain knowledge, and continuous learning. It's a challenging but rewarding field that offers many opportunities for personal and professional growth.

Step-by-step guide to building a machine learning model

 

1. Define the problem: Start by defining the problem you want to solve. What is the goal of the project, and how will the machine learning model help you achieve it?

2. Collect data: Once you have defined the problem, you will need to collect data. The data should be relevant to the problem you are trying to solve, and it should be representative of the population you are targeting.

3. Clean and preprocess the data: The data you collect may not be in the format you need for your machine learning model. You will need to clean and preprocess the data to get it into the right format.

4. Explore the data: Once the data is in the right format, explore it to gain insights. This may involve visualizing the data, computing descriptive statistics, or running simple machine learning models.

5. Choose a machine learning algorithm: Based on your exploration of the data, choose a machine learning algorithm that is appropriate for the problem you are trying to solve. There are many different algorithms to choose from, so you may need to experiment with several to find the best one.

6. Train the model: Once you have chosen an algorithm, train the model on your data. This involves feeding the data into the algorithm and letting it learn from the data.

7. Evaluate the model: Once the model is trained, evaluate it to see how well it performs. This may involve testing the model on a separate set of data or using cross-validation techniques.

8. Optimize the model: If the model does not perform well, you may need to optimize it. This may involve tweaking the parameters of the algorithm, collecting more data, or choosing a different algorithm.

9. Deploy the model: Once the model is optimized and performs well, deploy it in a production environment. This may involve integrating it with other software systems, setting up monitoring and logging, and creating user interfaces for the model.

10. Monitor and maintain the model: Once the model is deployed, you will need to monitor it to ensure it continues to perform well. This may involve tracking performance metrics, retraining the model with new data, and updating the model as necessary.

Overall, building a machine learning model is an iterative process that requires constant experimentation, optimization, and refinement. By following these steps, you can build a model that solves your problem and delivers value to your organization.

AspNet Core Background Task

 public class ScheduledService: IHostedService

{

    private readonly CrontabSchedule _crontabSchedule;

    private DateTime _nextRun;

    private const string Schedule = "0 0 23 * * *"; // run day at 11 pm

    public ScheduledService()

    {

        _crontabSchedule = CrontabSchedule.Parse(Schedule, new CrontabSchedule.ParseOptions{IncludingSeconds = true});

        _nextRun = _crontabSchedule.GetNextOccurrence(DateTime.Now);

    }

    public Task StartAsync(CancellationToken cancellationToken)

    {

        Task.Run(async () =>

        {

            while (!cancellationToken.IsCancellationRequested)

            {

                await Task.Delay(UntilNextExecution(), cancellationToken); // wait until next time

              //execute some task

                _nextRun = _crontabSchedule.GetNextOccurrence(DateTime.Now);

            }

        }, cancellationToken);

        return Task.CompletedTask;

    }

    private int UntilNextExecution() => Math.Max(0, (int)_nextRun.Subtract(DateTime.Now).TotalMilliseconds);

    public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask;

}

Cron Expressions refer

https://github.com/atifaziz/NCrontab

Elastic Search query_string Query

Query string query:

query_string query to create a complex search that includes wildcard characters searches across multiple fields

This query uses syntax to parse and split the provided query string based on operators, such as AND or NOT

Sample Queries:

GET /_search

{

  "query": {

    "query_string": {

      "query": "(new york city) OR (big apple)",

      "default_field": "content"

    }

  }

}

GET /_search

{

  "size": 10,

    "sort": [

    {

      "date": {

        "order": "desc"

      }

    }

  ],

  "query": {

    "query_string": {

      "query": "(IP:\"\" AND URI:*API*)"

    }

  }

}

Match

GET /_search

{

  "query": {

         "match" : {

            "URI" : "/index.html"

        }

  },

  "size": 10,

  "sort": [

    {

      "date": {

        "order": "desc"

      }

    }

  ]

}

Term

GET /_search

{

  "size": 10000,

  "sort": [

    { "date": "desc" }

  ], 

  

"query": {

    "bool": {

        "must" : [

          {

           "term": {

             "name": "Bob"

           }

         },

         {

           "term": {

             "gender": "male"

           }

         }

       ]

     }

   }

}

Machine Learning Week-1

What is Machine Learning?

A Computer program is said to be learn from experience E with respect to some class of tasks T and performance P, if its performance at tasks in T ,as measured by P, improves with experience E. 

--Tom Mitchell

Example: playing checkers.

E=the experience of playing many games of checkers.

T= task of playing checkers

P=the probability that the program will win the next game.

In general ,any machine learning problem can be assigned to one of two broad classifications:

Supervised learning,Unsupervised learning.

Get executed queries history Sql

SELECT  deqs.last_execution_time,dest.text
FROM    sys.dm_exec_query_stats AS deqs
        CROSS APPLY sys.dm_exec_sql_text(deqs.sql_handle) AS dest
WHERE   deqs.last_execution_time > '05/11/2017 11:00' -- check date
        AND dest.text LIKE '%querytomatch%' -- optional
        AND dest.dbid = DB_ID('dbname')
        order by deqs.last_execution_time desc

Difference between J2EE,J2SE & J2ME

J2SE stands for Java 2 standard edition and is normally for developing desktop applications, forms the core/base API.

J2EE stands for Java 2 enterprise edition for applications which run on servers, for example web sites.

J2ME stands for Java 2 micro edition for applications which run on resource constrained devices (small scale devices) like cell phones, for example games.

Netbeans is an IDE (Integrated development environment) developed in Java which eases your job of application development.

Centralized Asynchronous Auditing with Service Broker

Service Broker is a new feature in SQL Server 2005. It is an integrated part of the database engine and it provides queuing and reliable direct asynchronous messaging between SQL Server 2005 instances only. In the future this is planned to be extended to non-SQL Server instances. This article shows how to use Service Broker and triggers to capture data changes.

How Service Broker works

What Service Broker does is it talks or converses with other service brokers. It does that by exchanging messages in a dialog conversation between two service brokers. Imagine two people talking to each other. The words they exchange are messages and their conversation is a dialog. To fully understand the basics we must become familiar with the terminology and what does what. I will show only basic commands. More info can be found in SQL Server Help better known as Books Online.

Commands are explained in the order of needed object creation for Service Broker conversations.

The basis of everything is a message type. A message type defines the validation of the XML message that will be performed.

The general syntax is:

CREATE MESSAGE TYPE message_type_name
 [ AUTHORIZATION owner_name ]
 [ VALIDATION = { 
NONE | EMPTY | WELL_FORMED_XML | 
VALID_XML WITH SCHEMA COLLECTION 
schema_collection_name 
} ]

This is how to create a simple message type with validation that conforms to well formed XML:

CREATE MESSAGE TYPE [//Audit/Message] VALIDATION = WELL_FORMED_XML

Next step is to create a message type contract. This contract defines which message types are allowed in a conversation. For example, if we take our two people talking from before, a contract means that they are only allowed to talk about sports. Anything else is rejected by both persons as garbage.

The general syntax is:

CREATE CONTRACT contract_name
 [ AUTHORIZATION owner_name ]
 ( { { message_type_name | [ DEFAULT ] }
 SENT BY { INITIATOR | TARGET | ANY } 
} [ ,...n] ) 

In the SENT BY part we specify which message is allowed to be sent from each point of the conversation. So we can specify that the initiator can talk about sports and women, while the target can only talk about sports.

This is how to create a simple contract type with previously created message type that can be only sent by the initiator:

CREATE CONTRACT [//Audit/Contract] ([//Audit/Message] SENT BY INITIATOR)

Next comes a queue. A queue holds every message received by each point in the conversation. Each point of conversation has it's own queue in which the received messages are waiting for processing

The general syntax is:

CREATE QUEUE [ database_name. [ schema_name ] . | schema_name. ] queue_name

This is how to create a simple queue which takes:

• a stored procedure name to execute when a new message arrives in the queue
• maximum number of concurrently running stored procedures (for very busy queues)
• and user under whose context the procedure will be run

CREATE QUEUE dbo.TargetAuditQueue 
 WITH STATUS=ON, 
 ACTIVATION ( 
 PROCEDURE_NAME = usp_WriteAuditData, -- sproc to run when the queue receives a message
 MAX_QUEUE_READERS = 50, -- max concurrently executing instances of sproc
 EXECUTE AS 'dbo' );

Every queue is associated with a service. A service exposes the functionality of contracts associated with the service to other contracts. It defines which message types the associated queue will receive. Other types are rejected. If no contract is specified then that service can only be an initiator of the conversation so we have to specify which queue it will use and which contrats are allowed.

The general syntax is:

CREATE SERVICE service_name
 [ AUTHORIZATION owner_name ]
 ON QUEUE [ schema_name. ]queue_name
 [ ( contract_name | [DEFAULT] [ ,...n ] ) ]

This is how to create a simple service with previously created contract on our previously created queue with dbo authorization:

CREATE SERVICE [//Audit/DataWriter] 
 AUTHORIZATION dbo 
 ON QUEUE dbo.TargetAuditQueue ([//Audit/Contract])

Building the Centralized Asynchronous Auditing System

Now that we're familiar with the basics of Service Broker we can go on with building our auditing system. Auditing is the means of tracking changes of your data. It provides you with a log of who did what when. In the US and EU it is also required by law for sensitive data.

Auditing is usually done by inserting changed data into the accompanying audit table in the trigger of the source table. Or it can be a part of the Update, Delete and Insert stored procedures in which case we don't need triggers. Simplest way to implement auditing is for every table to have another audit table with the same structure. The downside is that these audit tables grow very fast. This increases the database size and backup/restore times which is a negative side effect. With a lot of tables this becomes cumbersome an hard to maintain. Imagine the work you have to do if you have 10 databases with each having 40 tables. That's 10 x 40 x 2 = 800 tables you have to create. And those are small databases table-wise.

Another option is to use third-party Log Readers but they aren't fun, are they? :)

I went about it differently. I wanted to have only one database that will hold all my audited data from every database I have on the server. This way my other databases would be free of bloated audited data. If i wanted to query the audited data i ccould simply select from one table that holds everything. I also wouldn't want this scenario to impact my performance. How to implement this? Service Broker to the rescue.

Service Broker's reliable asynchronous messaging was the perfect solution. I still used triggers but the technique can be easily used in stored procedures and OUTPUT clause. I created two databases each with its own Service Broker. The auditing database is called MasterAuditDatabase and the database to be audited is called TestDb1. Both databases have SET TRUSTWORTHY ON which enables us to use cross-database service broker communication without the use of certificates. Each database has its own error table that holds errors that happen in Service Broker communication. Yes, they may happen :)

When an insert, update or delete happens our audit trigger simply takes appropriate data from the inserted and deleted pseudo tables, turns it into our well formed XML message and uses Service Broker to send it to another Service Broker in the Master Audit Database which then saves it to our auditing table. Every time a new message arrives to our target queue, a stored procedure is executed that inserts the queued message into the audit table.

For busy systems only one stored procedure couldn't possibly cope with all incoming messages. That is why we set the MAX_QUEUE_READERS = 50. To allow 50 concurrent (parallel) queue readers. Because this is a completely asynchronous operation there's no impact on performance on the initiator end. The triggers return immediately.

The code can be downloaded and is well documented so I hope it speaks for itself. The code is broken into two scripts.

• The Master Audit Database (the database that holds all audited information) and its Service Broker infrastructure
• Sample audited database and its Service Broker infrastructure

Note: You will need to copy the GUID returned by the first script and use it in the second script.

This method of auditing proved to be very good with no noticeable impact on performance. Of course the master audit database grows fast and horizontal partitioning will be surely needed. It can also be used with multiple servers where one server serves as keeper of audited data from other servers. To do this we'd have to create TCP/IP endpoints and create users, certificates and other security measures which I will show how to do in the next article.

Conclusion

Service Broker is a great addition to SQL Server and it's use has just barely scratched the surface. The future is leaning to service oriented architecture (SOA) and loosly coupled applications. With the coming of .Net 3.5 and adoption of Windows Communication and Workflow Foundations, Service Broker will fit in nicely. So learn it.

Ref:

http://www.sqlteam.com/article/centralized-asynchronous-auditing-with-service-broker

Interview Questions

.Net Framework :

1. CLR 
2. MSIL
3. CTS
4. CLS
5. Garbage Collector its Types
6. How to Programmatically Invoke GC 

C# Interview Questions : 

1. Explain OOPS Concepts & Why to use them.
2. Can we use abstract class instead of interface? If yes then give example of each where to use interface and where to use abstract class. 
3. How Exceptions handled in C# and its types
4. Why do we not simply use delegates and scrap events?
5. What is Private Constructor? 
6. What is use of yield operator in C#?
7. Difference between ref and out
8. Diff.between constant and read only.
9. Explain new features in c# 4.0 dynamic, Co and Contra Variance?
10. Difference between Var and dynamics
11. Explain Reflection concept why to use .

C# Architecture Level :

1. Design Patterns Why to Use and Explain  its types.
2. Solid Principles

Asp.Net Questions:

1. Difference between HTTP Module and HTTP Handler? Asp.net page Life cycle? In which event we can modify session data initially?
2. Session Management

Linq Questions:

1. What is difference between IQueryable and IEnumerable?
2. Difference between Func, Action and predicate? They are delegate
3. How you debug LINQ query

Jquery/Javascript:

1. How you will debug JQuery request? If javascript disable then JQuery will work?
2. How you will create asynchronous request in JQuery? Syntax?
3. Give me scenario where you will use MVC rather than asp.net and MVVM or MVP?

MVC :

Unit Testing:

WCF Questions : 

1. How to do exception handling in WCF?
2. What is WCF throttling?

Web API:

1. What is Web Api & Why to Use.
2. WCF vs Web Api

SQL Questions:

1. What is Merge statement in SQL server? Sparse column? CTC feature?
2. Difference between RANK and DENS Rank? 
3. Which are isolation levels in SQL? Difference between snapshot and serializable?
4. Agile and scrum methodology? 

A brief understanding of pointers in C

The computer’s memory is a sequential collection of storage cells. Each cell, commonly known as a byte, has a number called address associated with it. Typically, the addresses are numbered consecutively, starting from zero. The last address depends on the memory size. A computer system having 64k memory will have its last address as 65,535.

Whenever we declare a variable in our programs, the system allocates somewhere in the memory, an appropriate location to hold the value of the variable. This location will have its own address number. Consider the following example:

The above statement int var creates a location in the memory to hold integer value. That location will have an address for example assume it is 5000. The statement var = 200 stores the value 200 at the location whose address is 5000. So, in our example, var is the variable name, 200 is the value stored in var and 5000 is the address of the memory location containing the variable var.

So, we can access the value 200 either by using the variable name var or by using the address of the memory location which is 5000. We can access using the second method by storing the address in another variable. This concept of storing memory address in a variable and accessing the value available at that address is known as a pointer variable. Since the pointer is also variable, it will also have a memory address just like any other variable.

Pointer Constants

The memory addresses within a computer are referred to as pointer constants.

Pointer Values

We cannot assign the memory addresses directly to a variable. We can only get the address of a variable by using the address operator (&). The value thus obtained is known as pointer value.

Pointer Variables

Once we obtain the pointer value, we can store it in a variable. Such variable which stores the memory address is known as a pointer variable.

Accessing the Address of a Variable

The actual location of a variable in memory is system dependent. A programmer cannot know the address of a variable immediately. We can retrieve the address of a variable by using the address of(&) operator. Let’s look at the following example:

1 2 3

int a = 10; int *p; p = &a;

In the above example, let the variable a is stored at memory address 5000. This can be retrieved by using the address of operator as &a. So the value stored in variable p is 5000 which is the memory address of variable a. So, both the variable a, and p point to the same memory location.

Declaring and Initializing Pointer Variables

Declaration

In C, every variable must be declared for its type. Since pointer variables contain addresses that belong to a specific data type, they must be declared as pointers before we use them. The syntax for declaring a pointer is as shown below:

1	datatype   *pointer–name;

This tells the compiler three things about the variable pointer-name. They are:

1. The asterisk (*) tells that the variable pointer-name is a pointer variable.
2. pointer-name needs a memory location.
3. pointer-name points to a variable of type datatype. 

Consider the following example for declaring pointers:

1 2	int  *p; float  *p;

Initialization

After declaring a pointer variable we can store the memory address into it. This process is known as initialization. All uninitialized pointers will have some unknown values that will be interpreted as memory addresses. Since compilers do not detect these errors, the programs with uninitialized pointers will produce erroneous results.

Once the pointer variable has been declared, we can use the assignment operator to initialize the variable. Consider the following example:

1 2 3

int  var = 20; int *p; p = &var;

In the above example, we are assigning the address of variable var to the pointer variable p by using the assignment operator. We can also combine the declaration and initialization into a single line as shown below:

1 2	int  var = 20; int *p = &var;

Accessing a Variable Using Pointer

After declaring and initializing a pointer variable, we can access the value of the variable pointed by the pointer variable using the * operator. The * operator is also known as indirection operator ordereferencing operator. Consider the following example:

1 2 3

int var = 20; int *p = &var; printf(”Var = %d”, *p);

In the above example, we are printing the value of the variable var by using the pointer variable palong with the * operator.