Microsoft SQL server - Find size of the Log files

Below script will help you to find the size of the log files in your db (data file,log file etc)
you can put the name of the database replacing "yourDb"
and if you need to get all the files details of the all the databases, just omit the filter where caluse


SELECT DB_NAME(database_id) AS DatabaseName,
Name AS Logical_Name,
Physical_Name, (size*8)/1024 SizeMB
FROM sys.master_files
WHERE DB_NAME(database_id) = 'yourDb'
GO

REST webservice - Apache CXF converts String to JSON incorrectly

I have created a simple REST webservice and it returns a String value from Response. But browser givesan error and nothing shows from tomcat side.


Error shown in font end: This is basically generic error

=======================

This page contains the following errors:

error on line 1 at column 1: Document is empty

Below is a rendering of the page up to the first error.

=============================

below is the coding for my webservice

------------------------------------------------------------------------------

@Path("/")

@WebService(name = "mailService", targetNamespace = "http://localhost/cxf-rest/example")

public interface MailService {

@GET

@Produces({ MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON })

@Path("/testMail")

public Response sendTestMail(@QueryParam("env") String env);

}

--------------------------------------------------------------------

----------------------------------------------------------------------------------------

public class MailServiceImpl implements MailService {

@Override

public Response sendTestMail(String env) {

// TODO Auto-generated method stub

return Response.ok("good").build();

}

}

----------------------------------------------------------------------------------------

below is the definitions for 'beans.xml"

------------------------------------------------------------------------------------------

<jaxrs:server id="mailService" address="/mailservices">

<jaxrs:providers>

<bean class="org.codehaus.jackson.jaxrs.JacksonJsonProvider" />

</jaxrs:providers>

<jaxrs:serviceBeans>

<ref bean="mailServiceImpl" />

</jaxrs:serviceBeans>

<jaxrs:extensionMappings>

<entry key="xml" value="application/xml" />

<entry key="json" value="application/json" />

</jaxrs:extensionMappings>

</jaxrs:server>

<bean id="mailServiceImpl" class="com.rest.cxfrestservice.service.MailServiceImpl" />

-----------------------------------------------------------------------------------------------

Reason

Jaxb does not know how to convert String to Json.

you have few solutions

Solution 1:

change response type to "text/plain"

----------------------------------------------------------------------------------------

@Path("/")

@WebService(name = "mailService", targetNamespace = "http://localhost/cxf-rest/example")

public interface MailService {

@GET

@Produces("text/plain")

@Path("/testMail")

public Response sendTestMail(@QueryParam("env") String env);

}

----------------------------------------------------------------------------------------

this will solve the error. But output will be a plain text.

Solution 2 : 
Use wrapper

----------------------------------------------------------------------------------------

@XmlRootElement(name = "statusMessage")
public class StatusMessage {

private String message;

public String getMessage() {
return message;
}

public void setMessage(String message) {
this.message = message;
}

}

----------------------------------------------------------------------------------------

new Service Impl class

---------------------------------------------------------------------------------------

@Path("/")
@WebService(name = "mailService", targetNamespace = "http://localhost/cxf-rest/example")
public interface MailService {

@GET
@Produces({ MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON })
@Path("/sendMail")
public Response sendRefreshMail(@QueryParam("env") String env);

}

------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------
public class MailServiceImpl implements MailService {

public Response sendRefreshMail(String env) {
if (env == null) {
return Response.status(Response.Status.BAD_REQUEST).build();
}
StatusMessage statusMessage = new StatusMessage();
statusMessage.setMessage(mailSao.sendEmail(env));

return Response.ok(statusMessage).build();
}


}

----------------------------------------------------------------------------------------


This will return response as needed. below is xml output

<statusMessage>
<message>SUCCESS</message>
</statusMessage>

below link is a good one

http://stackoverflow.com/questions/8339868/why-apache-cxf-converts-string-to-json-incorrectly

Loading properties from default property file location in maven project

Sometimes loading the property from file becomes a headache to java developers and it starts giving some awful errors. Specially finding the correct path for the property file.
As per maven property files will be placed inside "resources" folder and these files are copied into 'classes' folder inside "WEB-INF" (WEB-INF\classes)

below is a java code block works fines taking the given property file from default location. (default =WEB-INF\classes)

Here property file is loaded using class loader and you need Java 7 to use this logic.
With this you can easily get rid of pain came from  Java 1.5

======================================================

String resourceName = "config.properties"; // could also be a constant
ClassLoader loader = Thread.currentThread().getContextClassLoader();
Properties props = new Properties();
try (InputStream resourceStream = loader
.getResourceAsStream(resourceName)) {
props.load(resourceStream);
System.out.println(props);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}


=============================================================

Steps of building the Rest webservice

I have made a blog on creating a rest application. Just thought giving some steps to follow and check the application building process.

this is based on the link
http://cgenit.blogspot.com/2015/06/creating-webservice.html

1.       Create application “Rest” using maven plugin in eclipse.

2.       Deployed to tomcat and http://localhost:8080/Rest/ loading the home page

3.       http://cgenit.blogspot.com/2015/06/creating-webservice.html

4.       Apache CXF , spring , Jaxon, JAXB

5.       Added all the maven dependencies

6.       Build the project  mvn clean package , and redeploy in tomcat server and check home page http://localhost:8080/Rest/  loading fine

7.       Change the web.xml , Initially no entries except for

8.       You cannot just add changes to web.xml and test. It won’t work as web.xml refers beans and etc.

9.       Change web.xml

a.       Added classed

b.      Build and deploy   - not working

c.       My added “java” folder was not inside “Rest\src\main”

d.      I copied ”java” folder

e.      It was not working, cause I have named bean.xml where it should have been beans.xml – which is referred by the web.xml

f.        Make sure names are spelled correctly. Working fine

Adding or showing javascript , HTML code block inside blogger post

I have being trying several times to add html codes, java scripts inside a blogger post.
However using BLOCKQUOTE tag it was resolved.

Sometimes this might also give some headache, but for the time it gives me some hope to just to go on with my blogging.

so before the coding you are entering just add <blockquote>

Simple Angular Application to Call A Rest Webservice

Here I make an simple angular application to call a rest web service call

1. create sample rest web service. get an employee with id

http://cgenit.blogspot.com/2015/06/creating-webservice.html

2. let's create a simple web application calling this rest service

we call this rest service with

http://localhost:8080/Rest/rest/employeeservices/getemployeedetail?employeeId=1

3. We use the same "index.jsp" for this application
4. we create "hello.js" to put the angular based scripts
5. These script is called inside the "index.jsp"


the two files should be inside "webapp" folder . For this application path would be "Rest\src\main\webapp"

 6. build the project with maven and deploy the Rest.war file inside the tomcat and access the application using the url http://localhost:8080/Rest/

you will simply get an output as below.

Rest Application code structure

Below are the code snipets

hello.js

===========================================================

angular.module('demo', []).controller(

'Hello',

function($scope, $http) {

$http.get('http://localhost:8080/Rest/rest/employeeservices/getemployeedetail?employeeId=1').then(

function(response) {

$scope.employee = response.data;

});

});

=========================================================

index.jsp

============================================================

<!doctype html>

<html ng-app="demo">

<head>

<title>Hello AngularJS</title>

<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.4.3/angular.min.js"></script>

    <script src="hello.js"></script>

</head>

<body>

<div ng-controller="Hello">

<p>The ID is  = {{employee.employeeId}}</p>

<p>First name = {{employee.firstName}}</p>

<p>Last name  = {{employee.lastName}}</p>

<p>Date of join = {{employee.dateOfJoining}}</p>

<p>Department = {{employee.department}}</p>

<p>Email{{employee.email}}</p>

</div>

</body>

</html

============================================================

SQL Server :Link Server Test connection fail

Issue:
I have created a link server in MS SQL server.
Connect to database with "windows  Auhtentication"  - Connection successful
Conneect to database server with " SQL server Auhtentication -  Test connection fail

most proabably you will get an error like below


Cannot create an instance of OLE DB provider "OraOLEDB.Oracle" for linked server "YourLinkServerName". (Microsoft SQL Server, Error: 7302)

Solution : 
Enable "Allow inprocess" to "OraOLEDB.Oracle"

How to do this
---------

Server Objects --> Linked Servers -->Providers --> OraOLEDB.Oracle


tick Enable "Allow inprocess"

make sure you have link server using this provider

Maven jar dependencies are displayed outside `Maven Dependencies` view

showing the all jars listed in eclipse is a real headache. let's see how to get rid of that.

jars shown outside

1.  Just delete the project from the eclipse. (do not delete from disk, if you not sure about this - better to backup below delete)
2.  go to folder structure of the project and delete generated files. (".settings" folder, "target" folder, "classpath" file, ".project" file ) 

delete generated folders

     3. Import project using wizard  - existing maven project. You will get rid of the issue.

import again as a existing maven project

issue sorted - libraries are not shown now

Microsoft SQL server - Restore dates

It's handy stuff if you can find the refresh dates of a database. All details are inside db server and it's a matter of finding the correct query.

Below query will help to get the required information on refresh/ db restoration

SELECT [rs].[destination_database_name], 
[rs].[restore_date], 
[bs].[backup_start_date], 
[bs].[backup_finish_date], 
[bs].[database_name] as [source_database_name], 
[bmf].[physical_device_name] as [backup_file_used_for_restore]
FROM msdb..restorehistory rs
INNER JOIN msdb..backupset bs
ON [rs].[backup_set_id] = [bs].[backup_set_id]
INNER JOIN msdb..backupmediafamily bmf 
ON [bs].[media_set_id] = [bmf].[media_set_id] 
ORDER BY [rs].[restore_date] DESC

Few Good Youtube links for Understand Finacial Markets

The Causes and Effects of the 2008 Financial Crisis
https://www.youtube.com/watch?v=N9YLta5Tr2A

Economic Schools of Thought: Crash Course Economics

Copying logins from one Sql server to another SQL server 2012

when you restoring data base you may need to copy the users also. sql server gives a easy way of scripting out this process using two sps.

• sp_help_revlogin 
• sp_hexadecimal

Steps:

1. create two sps. naming " sp_help_revlogin " and "sp_hexadecimal "
2. script out the user details by running "sp_help_revlogin"
1.       exec sp_help_revlogin
3. If you need to run only for specific user run sp with user name.
1.      exec sp_help_revlogin 'Alex'
4. once you run scripts , you will get the script to be run on the other server. It's just a matter of copying it and run on the server you need to create/copy users 

For more information refer below links
https://support.microsoft.com/en-us/kb/918992
https://support.microsoft.com/en-us/kb/246133


Below are the script for two sps


-------------------------------start script----------------------------------------------------------------

USE master
GO
IF OBJECT_ID ('sp_hexadecimal') IS NOT NULL
  DROP PROCEDURE sp_hexadecimal
GO
CREATE PROCEDURE sp_hexadecimal
    @binvalue varbinary(256),
    @hexvalue varchar (514) OUTPUT
AS
DECLARE @charvalue varchar (514)
DECLARE @i int
DECLARE @length int
DECLARE @hexstring char(16)
SELECT @charvalue = '0x'
SELECT @i = 1
SELECT @length = DATALENGTH (@binvalue)
SELECT @hexstring = '0123456789ABCDEF'
WHILE (@i <= @length)
BEGIN
  DECLARE @tempint int
  DECLARE @firstint int
  DECLARE @secondint int
  SELECT @tempint = CONVERT(int, SUBSTRING(@binvalue,@i,1))
  SELECT @firstint = FLOOR(@tempint/16)
  SELECT @secondint = @tempint - (@firstint*16)
  SELECT @charvalue = @charvalue +
    SUBSTRING(@hexstring, @firstint+1, 1) +
    SUBSTRING(@hexstring, @secondint+1, 1)
  SELECT @i = @i + 1
END

SELECT @hexvalue = @charvalue
GO
 
IF OBJECT_ID ('sp_help_revlogin') IS NOT NULL
  DROP PROCEDURE sp_help_revlogin
GO
CREATE PROCEDURE sp_help_revlogin @login_name sysname = NULL AS
DECLARE @name sysname
DECLARE @type varchar (1)
DECLARE @hasaccess int
DECLARE @denylogin int
DECLARE @is_disabled int
DECLARE @PWD_varbinary  varbinary (256)
DECLARE @PWD_string  varchar (514)
DECLARE @SID_varbinary varbinary (85)
DECLARE @SID_string varchar (514)
DECLARE @tmpstr  varchar (1024)
DECLARE @is_policy_checked varchar (3)
DECLARE @is_expiration_checked varchar (3)

DECLARE @defaultdb sysname
 
IF (@login_name IS NULL)
  DECLARE login_curs CURSOR FOR

      SELECT p.sid, p.name, p.type, p.is_disabled, p.default_database_name, l.hasaccess, l.denylogin FROM 
sys.server_principals p LEFT JOIN sys.syslogins l
      ON ( l.name = p.name ) WHERE p.type IN ( 'S', 'G', 'U' ) AND p.name <> 'sa'
ELSE
  DECLARE login_curs CURSOR FOR


      SELECT p.sid, p.name, p.type, p.is_disabled, p.default_database_name, l.hasaccess, l.denylogin FROM 
sys.server_principals p LEFT JOIN sys.syslogins l
      ON ( l.name = p.name ) WHERE p.type IN ( 'S', 'G', 'U' ) AND p.name = @login_name
OPEN login_curs

FETCH NEXT FROM login_curs INTO @SID_varbinary, @name, @type, @is_disabled, @defaultdb, @hasaccess, @denylogin
IF (@@fetch_status = -1)
BEGIN
  PRINT 'No login(s) found.'
  CLOSE login_curs
  DEALLOCATE login_curs
  RETURN -1
END
SET @tmpstr = '/* sp_help_revlogin script '
PRINT @tmpstr
SET @tmpstr = '** Generated ' + CONVERT (varchar, GETDATE()) + ' on ' + @@SERVERNAME + ' */'
PRINT @tmpstr
PRINT ''
WHILE (@@fetch_status <> -1)
BEGIN
  IF (@@fetch_status <> -2)
  BEGIN
    PRINT ''
    SET @tmpstr = '-- Login: ' + @name
    PRINT @tmpstr
    IF (@type IN ( 'G', 'U'))
    BEGIN -- NT authenticated account/group

      SET @tmpstr = 'CREATE LOGIN ' + QUOTENAME( @name ) + ' FROM WINDOWS WITH DEFAULT_DATABASE = [' + @defaultdb + ']'
    END
    ELSE BEGIN -- SQL Server authentication
        -- obtain password and sid
            SET @PWD_varbinary = CAST( LOGINPROPERTY( @name, 'PasswordHash' ) AS varbinary (256) )
        EXEC sp_hexadecimal @PWD_varbinary, @PWD_string OUT
        EXEC sp_hexadecimal @SID_varbinary,@SID_string OUT
 
        -- obtain password policy state
        SELECT @is_policy_checked = CASE is_policy_checked WHEN 1 THEN 'ON' WHEN 0 THEN 'OFF' ELSE NULL END FROM sys.sql_logins WHERE name = @name
        SELECT @is_expiration_checked = CASE is_expiration_checked WHEN 1 THEN 'ON' WHEN 0 THEN 'OFF' ELSE NULL END FROM sys.sql_logins WHERE name = @name
 
            SET @tmpstr = 'CREATE LOGIN ' + QUOTENAME( @name ) + ' WITH PASSWORD = ' + @PWD_string + ' HASHED, SID = ' + @SID_string + ', DEFAULT_DATABASE = [' + @defaultdb + ']'

        IF ( @is_policy_checked IS NOT NULL )
        BEGIN
          SET @tmpstr = @tmpstr + ', CHECK_POLICY = ' + @is_policy_checked
        END
        IF ( @is_expiration_checked IS NOT NULL )
        BEGIN
          SET @tmpstr = @tmpstr + ', CHECK_EXPIRATION = ' + @is_expiration_checked
        END
    END
    IF (@denylogin = 1)
    BEGIN -- login is denied access
      SET @tmpstr = @tmpstr + '; DENY CONNECT SQL TO ' + QUOTENAME( @name )
    END
    ELSE IF (@hasaccess = 0)
    BEGIN -- login exists but does not have access
      SET @tmpstr = @tmpstr + '; REVOKE CONNECT SQL TO ' + QUOTENAME( @name )
    END
    IF (@is_disabled = 1)
    BEGIN -- login is disabled
      SET @tmpstr = @tmpstr + '; ALTER LOGIN ' + QUOTENAME( @name ) + ' DISABLE'
    END
    PRINT @tmpstr
  END

  FETCH NEXT FROM login_curs INTO @SID_varbinary, @name, @type, @is_disabled, @defaultdb, @hasaccess, @denylogin
   END
CLOSE login_curs
DEALLOCATE login_curs
RETURN 0
GO

-------------------end script------------------------------------------

file types in sql server database

There are three main file types

• MDF
• NDF
• LDF

MDF - Primary data file

Starting point of the database.This points to other files inside database.Every database has one primary data file.

All the data in the database objects are stored inside primary data file.(tables, stored procedures, triggers, views,....etc)

NDF - secondary data files

can have only one primary file and maintain any number of secondary data files

Optional - not necessary to have.

LDF - Log files

hold the all the log information

you can use this to recover database

size of log file depends on the log level defined by the database administrator

Should have at least one log file. ( can have multiples)

So basically you should have a primary data file and log file for each database.

mentioned extensions for the each category are optional and you can have any extension name instead of (.mdf, .ndf, .ldf) , but it is recommend to follow the standard. unless you have a necessity.

System Cannot Find the path specified - Restore database from back up file

you may have got the error "System Cannot Find the path specified " when restoring database from a back up file.
reason : sql server does not create the directory structure
solution: create the directory structure. you will get rid of the error

How to start SQL server or check whether SQL server installed or not

How to install, this is what has given in Microsoft site
https://technet.microsoft.com/en-us/library/ms190699.aspx

To start the default instance of SQL Server

1. On the Start menu, point to All Programs, point to Microsoft SQL Server 2008 R2, point to Configuration Tools, and then click SQL Server Configuration Manager.
2. In SQL Server Configuration Manager, in the left pane, click SQL Server Services.
3. In the details pane, right-click SQL Server (MSSQLServer), and then click Start.
   A green arrow on the icon next to the server name and on the toolbar indicates that the server started successfully.
4. Click OK to close SQL Server Configuration Manager.

To start a named instance of SQL Server

1. On the Start menu, point to All Programs, point to Microsoft SQL Server 2008 R2, point to Configuration Tools, and then click SQL Server Configuration Manager.
2. In SQL Server Configuration Manager, in the left pane, click SQL Server.
3. In the details pane, right-click the named instance of SQL Server, and then click Start.
   A green arrow on the icon next to the server name and on the toolbar indicates that the server started successfully.
4. Click OK to close SQL Server Configuration Manager.

To start an instance of SQL Server with startup options

1. On the Start menu, point to All Programs, point to Microsoft SQL Server 2008 R2, point to Configuration Tools, and then click SQL Server Configuration Manager.
2. In SQL Server Configuration Manager, in the left pane, click SQL Server.
3. In the details pane, right-click the instance of SQL Server, and then click Properties.
4. In the SQL Server (<instancename>) Properties dialog box, click the Advanced tab, and then click Startup Parameters.
5. At the end of the original text, in the Value column, type the startup parameters you want, and then click OK. Separate parameters with a semi-colon, for example, –c;-m.
6. Stop and restart SQL Server for the parameters to take effect.

To check whether SQL server installed ?

1. check SQL Server Configuration Manager. You can use the Search and type "SQL Server configuration Manager"

2. If it is not installed , you will probably won't find this. might get a screen like this.



3. If you have , you will see the results

4. click it. you will see the SQL server instance running.

Permission denied in accessing file system in Linux/Unix

Have you ever experienced this message "Permission denied" when trying to access a file system in Unix/linux.


I have created lower level user eg: noci and I have given full permission to read the folder structure

eg :
drwxr-xr-- 11 root root 4096 Aug  4 13:23 Folder

folder owner is root, but i have given read access to all as you can see.

But when tried to access folder it gives me " Permission denied"  message.


Solution:
Give the execution permission. Without execution permission user cannot go inside folder structure.

Create ping request and logging output as needed

Here i have created a small Java program to make a ping request and log the output from the ping request.

Program does below
1. make "ping" request to list of ips/hosts
2. log total output to a log file =  log4j-application-20160709124113837.log
3. Host name and corresponding Average time = log4j-ping-20160709124113853.csv

I am creating log for each run. for that I am appending time as a suffix for log name.For that I have created a custom file appender "CustomFileAppender.java" and it is being used inside log4j.properties.

1. For ping , I have created  java class "PingStatistics .java"
2. Inside main method I have created a list of hosts/ips. Here I have included three names. google.com,yahoo.com and bing.com
3. for each item in the list, or each host name i create the ping command to execute. eg: let's say we are going to ping google.com, then command is "ping google.com".
"public static String createCommandForPing(String ip)" method is creating the command for each host.
4. we execute the command after creating command.
  the method "public static void executeCommnd(String command, String ip) ".
 Here parameter "ip" is used for logging purposes.
 If i explain execution...
   we need to have a runtime. So we create a runtime in Java

-------------------------------------------------------
          Runtime r = Runtime.getRuntime();
--------------------------------------------------------
 Then we execute the command using the runtime. command will execute in seperate process and we can get the return process as below.
----------------------------------------------------------
          Process p = r.exec(command);
----------------------------------------------------------

Then we can get the output of the process. we can't directly get a string. We need to get the InputStream and convert to Buffer and get line by line. First we create a "BufferedReader" from the input stream of the process.
----------------------------------------------------------
         BufferedReader in = new BufferedReader(new InputStreamReader(
p.getInputStream()));
----------------------------------------------------------

Now e have the Buffer. Now we can read line by line of the output of the process. Or In other words , output of the execution of a ping command.

----------------------------------------------------------
String inputLine;
while ((inputLine = in.readLine()) != null) {
// System.out.println(inputLine);
log.info(inputLine);
pingResult += inputLine;
}
in.close();
----------------------------------------------------------

Remember to close the BufferedReader. "in.close(); "



Java code

======================================================================
package com.pingt;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;

import org.apache.log4j.Logger;

public class PingStatistics {
static final Logger log = Logger.getLogger(PingStatistics.class);
final static Logger log_pingReport = Logger.getLogger("reportsLogger");

public static String createCommandForPing(String ip) {
String pingCmd = "ping " + ip;

return pingCmd;
}

/**
* ip added for logging purposes.
*
* @param command
* @param ip
*/
public static void executeCommnd(String command, String ip) {
StringBuilder pingDetails = new StringBuilder();
pingDetails.append(ip);
pingDetails.append(",");

String pingResult = "";
try {
Runtime r = Runtime.getRuntime();
Process p = r.exec(command);

BufferedReader in = new BufferedReader(new InputStreamReader(
p.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
// System.out.println(inputLine);
log.info(inputLine);
pingResult += inputLine;
}
in.close();
System.out.println(pingResult);

String time = pingResult.substring(pingResult.indexOf("Average = "));
pingDetails.append(time);
log_pingReport.info(pingDetails);

} catch (IOException e) {
System.out.println(e);
}
}

public static String printPingDetailsHeader() {
StringBuilder header = new StringBuilder();
header.append("IP/Host Name");
header.append(",");
header.append("Average Time(ms)");

return header.toString();
}

public static void main(String[] args) {

List ipList = new ArrayList();
ipList.add("google.com");
ipList.add("yahoo.com");
ipList.add("bing.com");

log_pingReport.info(printPingDetailsHeader());
for (String ip : ipList) {
String command = createCommandForPing(ip);
executeCommnd(command,ip);
}

}
}

==================================================================

Below code for custom log Appender

==================================================================
package com.log;

import java.text.SimpleDateFormat;
import java.util.Date;

import org.apache.log4j.FileAppender;

public class CustomFileAppender extends FileAppender {

@Override
public void setFile(String fileName) {
if (fileName.indexOf("%timestamp") >= 0) {
Date d = new Date();
SimpleDateFormat format = new SimpleDateFormat("yyyyMMddHHmmssSS");
fileName = fileName.replaceAll("%timestamp", format.format(d));
}
super.setFile(fileName);
}
}

=================================================================

log4j.properties

==================================================================
# Root logger option
log4j.rootLogger=DEBUG, stdout, file

# Redirect log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%m%n

# Redirect log messages to a log file, support file rolling.
log4j.appender.file=com.log.CustomFileAppender
log4j.appender.file.File=C:\\logs\\log4j-application-%timestamp.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%m%n

log4j.category.reportsLogger=DEBUG, reportsLog
log4j.additivity.reportsLogger=false

log4j.appender.reportsLog=com.log.CustomFileAppender
log4j.appender.reportsLog.File=C:\\logs\\log4j-ping-%timestamp.csv
log4j.appender.reportsLog.layout=org.apache.log4j.PatternLayout
log4j.appender.reportsLog.layout.ConversionPattern=%m%n

===================================================================


Output for full log

===========  log4j-application-20160709124113837.log  =======================


Pinging google.com [74.125.200.102] with 32 bytes of data:
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46

Ping statistics for 74.125.200.102:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 49ms, Maximum = 49ms, Average = 49ms

Pinging yahoo.com [98.138.253.109] with 32 bytes of data:
Reply from 98.138.253.109: bytes=32 time=253ms TTL=46
Reply from 98.138.253.109: bytes=32 time=253ms TTL=46
Reply from 98.138.253.109: bytes=32 time=259ms TTL=46
Reply from 98.138.253.109: bytes=32 time=261ms TTL=44

Ping statistics for 98.138.253.109:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 253ms, Maximum = 261ms, Average = 256ms

Pinging bing.com [204.79.197.200] with 32 bytes of data:
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=48ms TTL=118

Ping statistics for 204.79.197.200:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 48ms, Maximum = 49ms, Average = 48ms

==================================================================

== output for host name/average  log4j-ping-20160709124113853.csv ====================

IP/Host Name,Average Time(ms)
google.com,Average = 49ms
yahoo.com,Average = 256ms
bing.com,Average = 48ms

====================================================================

Create ping request and logging output as needed

Here i have created a small Java program to make a ping request and log the output from the ping request.

Program does below
1. make "ping" request to list of ips/hosts
2. log total output to a log file =  log4j-application-20160709124113837.log
3. Host name and corresponding Average time = log4j-ping-20160709124113853.csv

I am creating log for each run. for that I am appending time as a suffix for log name.For that I have created a custom file appender "CustomFileAppender.java" and it is being used inside log4j.properties.

1. For ping , I have created  java class "PingStatistics .java"
2. Inside main method I have created a list of hosts/ips. Here I have included three names. google.com,yahoo.com and bing.com
3. for each item in the list, or each host name i create the ping command to execute. eg: let's say we are going to ping google.com, then command is "ping google.com".
"public static String createCommandForPing(String ip)" method is creating the command for each host.
4. we execute the command after creating command.
  the method "public static void executeCommnd(String command, String ip) ".
 Here parameter "ip" is used for logging purposes.
 If i explain execution...
   we need to have a runtime. So we create a runtime in Java
          Runtime r = Runtime.getRuntime();
 Then we execute the command using the runtime. command will execute in seperate process and we can get the return process as below.
          Process p = r.exec(command);

Then we can get the output of the process. we can't directly get a string. We need to get the InputStream and convert to Buffer and get line by line. First we create a "BufferedReader" from the input stream of the process.
         BufferedReader in = new BufferedReader(new InputStreamReader(
p.getInputStream()));

Now e have the Buffer. Now we can read line by line of the output of the process. Or In other words , output of the execution of a ping command.

String inputLine;
while ((inputLine = in.readLine()) != null) {
// System.out.println(inputLine);
log.info(inputLine);
pingResult += inputLine;
}
in.close();

Remember to close the BufferedReader. "in.close(); "



Java code

======================================================================
package com.pingt;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;

import org.apache.log4j.Logger;

public class PingStatistics {
static final Logger log = Logger.getLogger(PingStatistics.class);
final static Logger log_pingReport = Logger.getLogger("reportsLogger");

public static String createCommandForPing(String ip) {
String pingCmd = "ping " + ip;

return pingCmd;
}

/**
* ip added for logging purposes.
*
* @param command
* @param ip
*/
public static void executeCommnd(String command, String ip) {
StringBuilder pingDetails = new StringBuilder();
pingDetails.append(ip);
pingDetails.append(",");

String pingResult = "";
try {
Runtime r = Runtime.getRuntime();
Process p = r.exec(command);

BufferedReader in = new BufferedReader(new InputStreamReader(
p.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
// System.out.println(inputLine);
log.info(inputLine);
pingResult += inputLine;
}
in.close();
System.out.println(pingResult);

String time = pingResult.substring(pingResult.indexOf("Average = "));
pingDetails.append(time);
log_pingReport.info(pingDetails);

} catch (IOException e) {
System.out.println(e);
}
}

public static String printPingDetailsHeader() {
StringBuilder header = new StringBuilder();
header.append("IP/Host Name");
header.append(",");
header.append("Average Time(ms)");

return header.toString();
}

public static void main(String[] args) {

List ipList = new ArrayList();
ipList.add("google.com");
ipList.add("yahoo.com");
ipList.add("bing.com");

log_pingReport.info(printPingDetailsHeader());
for (String ip : ipList) {
String command = createCommandForPing(ip);
executeCommnd(command,ip);
}

}
}

==================================================================

Below code for custom log Appender

==================================================================
package com.log;

import java.text.SimpleDateFormat;
import java.util.Date;

import org.apache.log4j.FileAppender;

public class CustomFileAppender extends FileAppender {

@Override
public void setFile(String fileName) {
if (fileName.indexOf("%timestamp") >= 0) {
Date d = new Date();
SimpleDateFormat format = new SimpleDateFormat("yyyyMMddHHmmssSS");
fileName = fileName.replaceAll("%timestamp", format.format(d));
}
super.setFile(fileName);
}
}

=================================================================

log4j.properties

==================================================================
# Root logger option
log4j.rootLogger=DEBUG, stdout, file

# Redirect log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%m%n

# Redirect log messages to a log file, support file rolling.
log4j.appender.file=com.log.CustomFileAppender
log4j.appender.file.File=C:\\logs\\log4j-application-%timestamp.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%m%n

log4j.category.reportsLogger=DEBUG, reportsLog
log4j.additivity.reportsLogger=false

log4j.appender.reportsLog=com.log.CustomFileAppender
log4j.appender.reportsLog.File=C:\\logs\\log4j-ping-%timestamp.csv
log4j.appender.reportsLog.layout=org.apache.log4j.PatternLayout
log4j.appender.reportsLog.layout.ConversionPattern=%m%n

===================================================================


Output for full log

===========  log4j-application-20160709124113837.log  =======================


Pinging google.com [74.125.200.102] with 32 bytes of data:
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46
Reply from 74.125.200.102: bytes=32 time=49ms TTL=46

Ping statistics for 74.125.200.102:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 49ms, Maximum = 49ms, Average = 49ms

Pinging yahoo.com [98.138.253.109] with 32 bytes of data:
Reply from 98.138.253.109: bytes=32 time=253ms TTL=46
Reply from 98.138.253.109: bytes=32 time=253ms TTL=46
Reply from 98.138.253.109: bytes=32 time=259ms TTL=46
Reply from 98.138.253.109: bytes=32 time=261ms TTL=44

Ping statistics for 98.138.253.109:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 253ms, Maximum = 261ms, Average = 256ms

Pinging bing.com [204.79.197.200] with 32 bytes of data:
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=49ms TTL=118
Reply from 204.79.197.200: bytes=32 time=48ms TTL=118

Ping statistics for 204.79.197.200:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 48ms, Maximum = 49ms, Average = 48ms

==================================================================

== output for host name/average  log4j-ping-20160709124113853.csv ====================

IP/Host Name,Average Time(ms)
google.com,Average = 49ms
yahoo.com,Average = 256ms
bing.com,Average = 48ms

====================================================================

Creating a log for each run/Custom file appender and log different outputs details to different files

Here i explain how to create a log for each run. Basically by adding a custom appender.
next i need to log different details to different files. In simple log categories of details to different log files.

In my previous blog entry i have explained adding custom appender.
http://cgenit.blogspot.com/2016/07/writing-custom-file-appender-for-log4j_1.html

anyway I will explain again with steps

Step 1 - create java class for custom appender
Step 2 - add log4j entry for custom appender
Step 3 - add log4j entry for create another log for selected category
Step 4 -  used log4j category and write details to another log file


Step 1

======================================================
import java.text.SimpleDateFormat;
import java.util.Date;

import org.apache.log4j.FileAppender;

public class CustomFileAppender extends FileAppender {

@Override
public void setFile(String fileName) {
if (fileName.indexOf("%timestamp") >= 0) {
Date d = new Date();
SimpleDateFormat format = new SimpleDateFormat("yyyyMMddHHmmssSS");
fileName = fileName.replaceAll("%timestamp", format.format(d));
}
super.setFile(fileName);
}
}

=======================================================


Step 2 and Step3  - log4j.properties

========================================================
# Root logger option
log4j.rootLogger=DEBUG, stdout, file

# Redirect log messages to console
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%m%n
####FullLog################
log4j.appender.file=com.research.log.CustomFileAppender
log4j.appender.file.File=D:\\Log\\FullLog-%timestamp.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%m%n
######accountResult##############
log4j.category.accountResultLogger=DEBUG, accountResultLog
log4j.additivity.accountResultLogger=false
##
log4j.appender.accountResultLog=com.research.log.CustomFileAppender
log4j.appender.accountResultLog.File=D:\\Log\\accountResult-%timestamp.csv
log4j.appender.accountResultLog.layout=org.apache.log4j.PatternLayout
log4j.appender.accountResultLog.layout.ConversionPattern=%m%n
#####studentResultLog####
log4j.category.studentResultLogger=DEBUG, studentResultLog
log4j.additivity.studentResultLogger=false
##
log4j.appender.studentResultLog=com.research.log.CustomFileAppender
log4j.appender.studentResultLog.File=D:\\Log\\studentResult-%timestamp.csv
log4j.appender.studentResultLog.layout=org.apache.log4j.PatternLayout
log4j.appender.studentResultLog.layout.ConversionPattern=%m%n

==========================================================

Step 4

public class Logging {


final static Logger Log = Logger.getLogger(Logging.class);
final static Logger Log_accountResult = Logger.getLogger("accountResultLogger");
final static Logger Log_studentResult = Logger
.getLogger("studentResultLogger");

Log_accountResult.info("accountResult");
Log_studentResult.info("studentResult");

}

This will logging as per category in different files

Writing a custom File Appender for log4j

log4j is a powerful and very versatile tool. here is a code snippet to write a custom file Appender.
This way you can let the log4j to create a log file for each run.
This is the biggest advantage over custom appenders in log4j

Step 1

The way i desire, First we need to write a java class for making a custom appender.Let's call it "CustomFileAppender.java"

code will be as follows

================================================================
import java.text.SimpleDateFormat;
import java.util.Date;

import org.apache.log4j.FileAppender;

public class CustomFileAppender extends FileAppender {

@Override
public void setFile(String fileName) {
if (fileName.indexOf("%timestamp") >= 0) {
Date d = new Date();
SimpleDateFormat format = new SimpleDateFormat("yyyyMMddHHmmssSS");
fileName = fileName.replaceAll("%timestamp", format.format(d));
}
super.setFile(fileName);
}
}

=====================================================================

her we add the date and time in format "yyyyMMddHHmmssSS" after the end of file name. In otherwords as a suffix.

eg: then your log file name would be like  "log-2016063014401283"

I believe that everybody understand the numbers

for ease of everybody, i will explain that part also

the format we use "yyyyMMddHHmmssSS" and  example is "2016063014401283".
let's take part by part from left to right.

Date formatter	Description	example
yyyy	year	2016
MM	MM	06
dd	date	30
HH	hours	14
mm	minutes	40
Ss (simple)	Seconds in minute	12
SS  (capital)	Milliseconds	83

for more information  refer

https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

let's come back to logging.

Step 2

adding appender to log4j property file  "log4j.properties"

=============================================================

# Root logger option

log4j.rootLogger=DEBUG, stdout, file

# Redirect log messages to console

log4j.appender.stdout=org.apache.log4j.ConsoleAppender

log4j.appender.stdout.Target=System.out

log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

log4j.appender.stdout.layout.ConversionPattern=%m%n

####FullLog################

log4j.appender.file=com.research.log.CustomFileAppender

log4j.appender.file.File=D:\\Log\\mergeOperation-%timestamp.log

log4j.appender.file.layout=org.apache.log4j.PatternLayout

log4j.appender.file.layout.ConversionPattern=%m%n

==========================================================

explanation

you can see that "log4j.rootLogger"  has "file" in line number 1

# Root logger option

log4j.rootLogger=DEBUG, stdout, file

next line describe, how we handle the standard output when running program. eg: when you run the program in console. you can print the output as needed modifying this.This is not what we are looking for.

Creating appender for the log file, what we log actually.That is explained in the below set.

# Redirect log messages to console

log4j.appender.stdout=org.apache.log4j.ConsoleAppender

log4j.appender.stdout.Target=System.out

log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

log4j.appender.stdout.layout.ConversionPattern=%m%n

here is how we add the custom appender.

log4j.appender.file=com.research.log.CustomFileAppender

log4j.appender.file.File=D:\\Log\\newLog-%timestamp.log

log4j.appender.file.layout=org.apache.log4j.PatternLayout

log4j.appender.file.layout.ConversionPattern=%m%n

we have created file option using "CustomFileAppender.java " class. Give the name with the package except ".java"

log4j.appender.file=com.research.log.CustomFileAppender

Then we create the name of the log file in the specific destination we need.We are creating log file inside folder name "Log" in drive "D". name of the log file is "newLog.log". But as we need to ad an appender , the time as a suffix to the log name with hyphen, we give new name as  "newLog-%timestamp.log"

It will be like this

log4j.appender.file.File=D:\\Log\\newLog-%timestamp.log

next two lines describe the pattern. first you give the patter class (log4j.appender.file.layou) for the patter and after that you can decide the pattern of the log with "log4j.appender.file.layout.ConversionPattern" property.

log4j.appender.file.layout=org.apache.log4j.PatternLayout

log4j.appender.file.layout.ConversionPattern=%m%n