We want to send mails which are authenticated by a mail server, but the application can only send mails to a unauthenticated mail server. We achieve this end result of authenticated mails, by using postfix (a mail relay/forwarding tool ).

What is Postfix? Its a mail server program written by "Wietse Zweitze Venema" that started life at IBM Research as an alternative to the most popular linux "Sendmail" tool. Now he works at Google,  and continues to support Postfix.

Postfix attempts to be fast, easy to setup, and secure. After using postfix you may feel that it's just a sendmail tool, but from inside its completely different. Most Linux OS would have postfix installed by default, if not you can install it using yum or apt-get based on your OS. Before setting up postfix make sure you also have the mail tool installed. Lets try to go through the flow we want to achieve.



Configuration Steps (RedHat):
1. Install the tools
yum update && yum install postfix mailx

2. Setup Gmail Authentication
Create or modify a credentials file which will be used by Postfix to establish authentication with GMail or any other mail server.
Add your credentials to a file (these are the details of the "mail from" user)
echo  "[smtp.gmail.com]:587     starkadmin@gmail.com:password" >/etc/postfix/sasl_passwd

3. Secure this credentials file
chmod 600 /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd

4. Create the security configurations file and secure it
echo "smtp.gmail.com:587   encrypt" >/etc/postfix/tls_policy
postmap /etc/postfix/tls_policy

5. Fetch the remote server's signer certificate
Note: Its a one line command
echo  -n | openssl  s_client  -connect  gmail.com:443 | \
sed  -ne  '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'  >/etc/postfix/cacert.pem

6. Create the mail re-write policy file
The file /etc/postfix/smtp_header_checks contains rules to be used to rewrite the headers of the emails about to be sent.
This is the most important file in our case as it rewrites the sender so that it always matches our GMail account, starkadmin@gmail.com
No more 'Relaying disallowed' errors!
echo  "/^From:.*/   REPLACE  From:starkdmin@gmail.com" > /etc/postfix/smtp_header_checks

7. Configure postfix
Edit postfix configuration file /etc/postfix/main.cf with following values:
relayhost = smtp.gmail.com:587
smtp_header_checks = pcre:/etc/postfix/smtp_header_checks
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_tls_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_password
smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
smtpd_use_tls = yes
smtp_tls_CAfile = /etc/postfix/cacert.pem

8. Enable IMAP for gmail user's (eg: starkadmin@gmail.com) account.

9. Restart postfix service and send a sample mail
service restart postfix

10. Enable "Less Secure Apps" In GMail
By default, only the most secure sign-ins, such as logging in to GMail on the web, are allowed for your GMail account. To permit relay requests, log in to your GMail account and turn on Allow less secure apps. For more information, review the Google Support document
"Allowing less secure apps to access your account."

11. All DONE
Let's send a test mail to confirm all our efforts...
echo  "This mail is generated by the configuration steps from 'http://www.cubicrace.com'"  |  mail  -s  "Postfix configuration" "piyush@cubicrace.com"

Troubleshooting
For postfix log, see /var/log/maillog

Still not working, try this ...
In the file /etc/postfix/master.cf, I uncommented this line:
smtps  inet  n  -  -  -  -  smtpd

Unauthenticated mail server configuration using mail forwarding with authentication



Where Good Ideas come from ? ... by Steven Jhonson.
A Fantastic video filled with loads of real life examples and inspirational thoughts....


Good Ideas by Smarter people for a Smarter planet


Archives are the heart and history of any blog. While the recent blog posts appear on the blog's home page, older posts are harder to find. Thanks to the archiving feature, so that the older posts can be found online at any time in the future...


Archive


In WEB based applications RESTful APIs are used to manipulate data between the browser and the server. User may enter some data which is formatted to a valid JSON object (Java Script Object Notation) before sending it to the server for processing. However at the server side this data may be stored in an XML format, which means we need to now convert this data from JSON to XML and vice-versa while sending data from server to the browser.

Below is a screenshot showing a sample XML on the left side and the generated JSON on the right side:


Conversion of JSON to XML isn't straight forward since they don't share common aspects about their structure. One may write a parser to read JSON elements one by one and then generate an XML but its a tedious job. Thanks to the power of Java Architecture for XML Binding (JAXB) which can easily parsing of XML (based on its dtd - document type definition) ie: convert an XML string into a java object and vice-versa. However pure jaxb implemetation isin't sufficient to solve our problem. We need a third party library named eclipselink.jar (download from here) which has the jaxb implementation to convert XML to JSON and vice versa. Alternatively we can use 3 sub jars from the eclipselink.jar - org.eclipse.persistence.core-2.5.1.jar, org.eclipse.persistence.antlr-2.5.1.jar, org.eclipse.persistence.moxy-2.5.1.jar

Steps:
1. Generate java classes using the dtd file of the XML
/usr/java/jdk1.7.0_60/bin/xjc
-dtd /home/cubicrace/workspace/json2xml/samples/DatabaseInventory.dtd
-d /home/cubicrace/workspace/json2xml/src/
-p com.utilities.jaxb

parsing a schema...
compiling a schema...
com/utilities/jaxb/Administrator.java
com/utilities/jaxb/DatabaseAttributes.java
com/utilities/jaxb/DatabaseInventory.java
com/utilities/jaxb/DatabaseName.java
com/utilities/jaxb/ObjectFactory.java


2. Create a jaxb.properties file with the below content:
javax.xml.bind.context.factory=org.eclipse.persistence.jaxb.JAXBContextFactory

Ensure that this file exists at the same location as the .class files of the above generated java files.

3. Add the eclipselink.jar to the classpath or project's java build path

4. Write a simple class to perform the conversion.
Below is a working standlone class:


package com.utilities;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;

import org.eclipse.persistence.jaxb.MarshallerProperties;
import org.eclipse.persistence.jaxb.UnmarshallerProperties;

import com.utilities.jaxb.DatabaseInventory;

public class JsonXmlConverter {

 public static final String base = "/home/cubicrace/workspace";
 
 static String srcJsonFile = base + "/json2xml/samples/dbInventory.json";
 static String srcXmlFile = base +  "/json2xml/samples/dbInventory.xml";    
 static String convertedJsonFile = base + "/json2xml/samples/convertedFile.json";
 static String convertedXmlFile = base + "/json2xml/samples/convertedFile.xml";
 
 public boolean createFile(String filePath) {
  File file = new File(filePath);
  file.delete();
  try {
   return file.createNewFile();
  } catch (IOException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
   return false;
  }
 }
 
 private void jsonToXml(String jsonFile) throws Exception{
  File jsonObj = new File(jsonFile);
  JAXBContext jc = JAXBContext.newInstance(DatabaseInventory.class);
  Unmarshaller unmarshaller = jc.createUnmarshaller();
  unmarshaller.setProperty(UnmarshallerProperties.MEDIA_TYPE, "application/json");
  DatabaseInventory activity = (DatabaseInventory) unmarshaller.unmarshal(jsonObj);
  
  Marshaller xmlM = jc.createMarshaller();
  xmlM.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
  FileOutputStream fos = new FileOutputStream(new File(convertedXmlFile));
  xmlM.marshal(activity, fos);
  fos.close();
 }
 
 private void xmlToJson(String xmlFile) throws Exception{
  File xml = new File(xmlFile);
  JAXBContext jc = JAXBContext.newInstance(DatabaseInventory.class);
  Unmarshaller unmarshaller = jc.createUnmarshaller();
  DatabaseInventory activity = (DatabaseInventory) unmarshaller.unmarshal(xml);

  Marshaller marshaller = jc.createMarshaller();
  marshaller.setProperty( Marshaller.JAXB_FORMATTED_OUTPUT, true);
  marshaller.setProperty( MarshallerProperties.MEDIA_TYPE, "application/json");
  marshaller.setProperty( MarshallerProperties.JSON_INCLUDE_ROOT, true);
  FileOutputStream fos = new FileOutputStream(new File(convertedJsonFile));
  marshaller.marshal(activity, fos);
  fos.close();
 }
 
 public static void main(String[] args) {

  JsonXmlConverter jxc = new JsonXmlConverter();
    
  try {
   jxc.createFile(convertedJsonFile);
   System.out.println("Source XML: " +  srcXmlFile);
   jxc.xmlToJson(srcXmlFile);
   System.out.println("Converted Output JSON: " + convertedJsonFile);
   
   jxc.createFile(convertedXmlFile);
   System.out.println("Source JSON: " + srcJsonFile);
   jxc.jsonToXml(srcJsonFile);
   System.out.println("Converted Output XML: " + convertedXmlFile);
  } catch (Exception e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }
  
 }

}


Note: Ensure to update the workspace variable in the above java class before executing in your local environment.

If you see the below error, it means the jaxb.properties file (step2 above) is missing at the expected location or has incorrect property value pair.
javax.xml.bind.PropertyException: name: eclipselink.media-type value: application/json
at javax.xml.bind.helpers.AbstractMarshallerImpl.setProperty(AbstractMarshallerImpl.java:337)
at com.sun.xml.internal.bind.v2.runtime.MarshallerImpl.setProperty(MarshallerImpl.java:539)

Download the dtd, sample files, eclipselink.jar and source code as single zip: here

How to convert a given XML snippet to an equivalent JSON format and vice-versa

The RBI will soon be issuing ₹ 2000 denomination banknotes in the Mahatma Gandhi (New) Series, without the inset letter, bearing signature of Dr. Urjit R. Patel, Governor, Reserve Bank of India, and the year of printing '2016' printed on the reverse of the banknote.

The new denomination has picture of Mangalayan on the reverse, depicting the country’s first venture into the interplanetary space. The base colour of the note is magenta. The note has other designs like geometric patterns aligning with the overall colour scheme, both at the obverse and reverse.

RBI's website doesn't mention anything about the NGC chip embedded in the currency ... Seems like a rumour/fake news.

Certain rumours about the 2000 currency note:
  1. Apparently, the new notes will have nano-GPS chips installed on them, for tracking. These chips will reportedly allow satellites to track the currency, even at 120 metres below ground level.
  2. Further, rumours say that the nano-GPS chips work without any power source, which is quite interesting if true.
  3. Moreover, the satellites will reportedly track large amounts of money stored at any place, automatically, and pass on the information to the Income Tax department.
  4. The department can then track such currency and recover it, if it’s found to be illegal.
  5. Removing the nano-GPS will reportedly render the currency unusable in the first place. 

Tracking currency may not be true, WHY ? 

1. Real Fact:
World’s smallest fully integrated GPS receiver has been developed by OriginGPS Nano Spider, which measures 4 x 4 x 2.1 mm. Comparably, this device is really small (but not smaller than a currency note), and can open a new avenue of wearable devices – now this chip can track clothes, watches, electronic appliances and more. However thickness of 2.1mm is big to fit in a currency note which is 0.1-0.2 mm thick.

2. Common PHYSICS:
GPS satellites are located in the medium earth orbit which starts at 20,200 Kilometers from Earth's surface, so deflecting a signal that far without any energy is impossible (https://en.wikipedia.org/wiki/GPS_satellite_blocks). If this was an available technology, it would have been used in mobile phones and there was no need to use the A-GPS ( which uses your mobile's data connection and GPS sensor to detect your mobile's location )

3. Link to RBI's site for details about the ₹2000 note:
 https://rbidocs.rbi.org.in/rdocs/PressRelease/PDFs/PR1144EFECD860ED0D479D88AB8D5CA036FC35.PDF

4. A chip without any power source will be able to embed the currency series number to the received GPS signal and deflect it back ???  this is beyond imagination and against science laws ... 


Front Features (As seen on RBI website)
  1. See through register with denominational numeral 2000
  2. Latent image with denominational numeral 2000
  3. Denominational numeral २००० in Devnagari
  4. Portrait of Mahatma Gandhi at the centre
  5. Micro letters ‘RBI’ and ‘2000’ on the left side of the banknote
  6. Windowed security thread with inscriptions ‘भारत’, RBI and 2000 on banknotes with colour shift. Colour of the thread changes from green to blue when the note is tilted
  7. Guarantee Clause, Governor’s signature with Promise Clause and RBI emblem towards right
  8. Denominational numeral with Rupee Symbol, ₹2000 in colour changing ink (green to blue) on bottom right 
  9. Ashoka Pillar emblem on the right
  10. Mahatma Gandhi portrait and electrotype (2000) watermarks
  11. Number panel with numerals growing from small to big on the top left side and bottom right side
    For visually impaired
    Intaglio or raised printing of Mahatma Gandhi portrait, Ashoka Pillar emblem, bleed lines and identity mark
  12. Horizontal rectangle with ₹2000 in raised print on the right
    Seven angular bleed lines on left and right side in raised print 
Reverse Features (As seen on RBI website)
  1. Year of printing of the note on the left
  2. Swachh Bharat logo with slogan
  3. Language panel towards the centre
  4. Motif of Mangalayan
  5. Denominational numeral २००० in Devnagari

India - Nano GPS chip or NGC in currency


When you read the word snapshot what is the first thing that comes to your mind is , "A Photograph which preserves the best moments of your life". Technically snapshot is very much the same with the difference that it preserves the state of some digital resource. In VMware a disk "snapshot" is a copy of the VM's disk file (.vmdk) captured at a certain point in time. This snapshot preserves the disk file system and the files stored on it which can be of any type (including all the operating system files). So if something goes wrong with your virtual machine, you can restore it to a snapshot which was working previously.

One can also create snapshots for different versions/service-packs on an OS.
Hence snapshots can also be looked upon as version controlling mechanism at OS level. So if your computer was shut down abruptly or gets infected by virus, just revert to a snapshot.

So how do snapshots really work ? There's just one Thumb Rule to VMware's snapshot technology : "Snapshots only store the differences between the current state and the original state". It follows the copy-on-write scheme with every subsequent disk access. Lets try to understand what that means ...

Consider that you have a text file with the word "COMPUTING" stored in it.
This file is a sparse in nature : which means it spans across multiple blocks on the disk. Step 1 below demonstrates this scenario. The black lines indicate the links on to the stored data. For demonstration purpose lets consider that each block on disk has only one character.


Note : The blocks shown above contain only one character and is purely for example purpose. In real the block size could be of say 1MB or a sector on disk.
 
Now when you take a snapshot another file named Snapshot1.vmdk will be created. When you create a snapshot, any changes made on the original virtual disk image are not made on the original disk, but they are written to a new (snapshot) disk file. This action is very fast as there is no need to copy whole virtual disk image.

Thumb Rule : "While saving changed data blocks in a snapshot, all modified block will be saved first , followed by blocks which were deleted as compared to base disk blocks." As seen in Step 2 , blue block is linked at the end of the snapshot1.vmdk

Lets suppose that you take a snapshot after you have saved the word "COMPUTING" in the file. After the snapshot you modify the file by changing its last two blocks (letters N and G circled above) and clear the letter I. The new changed word is "COMPUTER" as show in the Step 2. The blue block above is nothing but an empty block created by deleting letter I. The blocks in RED represents the new snapshot1.vmdk disk which contains only the changed characters.

Thumb Rule : "While reading any file in current state read only the data accessible by first level links, irrespective of number of snapshots and original data of the file."

Reading the first links of the "Current State" disk from Step 2 (Green blocks) , word "COMPUTER" is retrieved and the size of snapshot1.vmdk is 3 blocks (2 filled and one empty). However since its a differential snapshot file its size is much less than original base disk (9 blocks).  Snapshot image size grows as you continue to change more and more data from your original virtual disk image (which remains untouched from the moment you took the snapshot).

Thumb Rule : "Size of a snapshot will always be less than the base disk, but in worst case it will be exactly the same size if all blocks were to be changed."


As seen in Step 3 , we now take another snapshot after saving the word "COMPUTER" in the file. On making more changes after snapshot2 , similar process is followed to create snapshot2.vmdk file. The new changed word in the file is  "CONTAINER". As a result now neither snapshot 1 nor the base file are written two, but are still referenced. Snapshot 2 will store the new changes as compared to snapshot 1. If you read the first level links of the green blocks from top to bottom , the word "CONTAINER" is read with the fact that only 5 letters are stored in snapshot2.

Below are the list of changes made to the file
--------------------------------------------------------------------------
Step1 - Base Disk  = COMPUTING
Step2 - Snapshot1 = COMPUTER
Step3 - Snapshot2 = CONTAINER

From the above scenarios its clear that taking snapshots in VMware involves only writing the differences in files changed from the time of the snapshot, not the complete virtual machine disk. This mechanism is similar to taking diff and patch in Unix, but in a more sophisticated way that diffs on a binary level with the knowledge of how a VMFS ( Virtual Machine File System ) is structured.

Now we have a clear idea about the Copy-On-Write Protocol - Every time a block from the base disk or previous snapshot is changed , copy it to the current delta or snapshot file, which implies that when you perform a snapshot restoration, it only has to rewrite the sectors that were modified since you took the snapshot. As a result snapshot revert is also super fast.

But the Question is, What happens when you revert to an older snapshot? The VM software throws away the contents of snapshot2.vmdk and starts over with  reading contents from snapshot1.vmdk. During this all the Blue links in Step3 are replaced with Green Links. ( this is a similar strategy followed in deleting a node from a linklist ... Pure programming stuff ). Note : Links to Snapshot1 and Base are not changed.

There are two more important aspects of Snapshot management : Discarding a snapshot and Merging a snapshot (reverting to a non-immediate parent snapshot).

A KB article from VMware on Snapshot management.
http://www.youtube.com/watch?feature=player_embedded&v=rj8ugLrmU-M

How do Virtual Machine Snapshots work in VMware

Steven Paul "Steve" Jobs (February 24, 1955 – October 5, 2011) was an American information technology entrepreneur and inventor. He was the co-founder, chairman, and chief executive officer (CEO) of Apple Inc.


























More about him: https://en.wikipedia.org/wiki/Steve_Jobs

12 Rules of Success by Steve Jobs


You would fire a HTTP or HTTPS request to a Remote Server (HTTP) to send data. Sending JSON data / XML data is different than sending file data.
A HTTP multipart request is a HTTP request that HTTP clients construct to send files and data over to a HTTP Server. It is commonly used by browsers and HTTP clients to upload files to the server.

Let me explain how a browser encodes an html form before sending to server.

When you a select a file to be uploaded via a browser, it creates a multipart (Multiple Parts) request which consists of file content and other supporting information like filesize, boundary string, content type, additional parameters etc. Instead of URL encoding the form parameters, the form parameters (including the file data) are sent as sections in a multipart document in the body of the request.

Here's what a Multipart Request looks like (Firebug View):



Transfer-Encoding chunked:
Chunked transfer encoding is a data transfer mechanism in which data is sent in a series of "chunks".

Content-Type     multipart/form-data; boundary=---------------------------204472851627279345251768965
The Content-Type "multipart/form-data" specifies how the form data set is encoded. Form-data is a sequence of "name / value" pairs defined inside 'form' section. Boundary is just random unique string used to mark separation between different form-data sequences. Let me show you what I mean:

Post Data:

For example in the above form, data set is:
name=filename
value=color-scheme-6656-main.png

name=filePath
value=lib/

name=uploadedfiles[]
value=binary encode filecontent, which is a image file

The encoding "multipart/form-data" says, message contains a series of parts each part contains a name-value pair in the "form data set". Each part is separated using the "boundary" string and is specified in the below format:

// New line
// New line
--<value-of-boundary>
Content-Disposition: form-data; name="<control-name>" [filename="name-of-the-file-if-control-is-file"]
[Content-Type: <The-content-type-of-the-uploded-file-if-control-is-file>]
// New line
Actual file content either as clear text of binary data depending on file type
// New line
// New line

After all parts have been added to the request body, we need to indicate the end of all parts with a string that is formed by appending and prep-ending the boundary attribute by two dashes (--). For example it is ---------------------------204472851627279345251768965-- as show in the last red box of the above screenshot.
Once the request is ready & fired with all the above data, the receiver (server) will analyse all the information from the request and should be able to save the file will all the content intact.

Securely upload files to a Remote Server using JAVA client code - Part 1

+