Friday, November 28, 2014

The Way To A New Project --- Becoming Effective

You move to a new project. Its exciting. It holds promise of something awesome (usually). You are ready to dive. You want to learn things fast. You want to contribute. You want to become effective. You want to be welcomed. And then maybe, become invaluable.

There are some tips I have learnt on how to navigate new projects effectively. I would look forward to hearing what you have to say about yours. And, I suspect there will be more.


  1. The more experience you get on multiple tools and technologies, the easier it is to move to newer projects with newer tech stacks. 
  2. The more you master one programming language, the more difficult you find adopting another language. Especially if you are a perfectionist.
  3. The more you expose yourself to various languages, the better you program in all previous languages.
  4. The quicker you understand the domain of the project, the quicker you understand the code of your project (unless people in your team like naming everything as “data” or “temp” or “value” or “count”, in which case you are doomed, and you should quit).
  5. Understand the database of your project. The database brings insights into relationships between entities. These relationships will remain permanent for a very long time, while the code on top that manipulates the data, will keep getting refactored. Once you get a handle on the data, the code becomes much easier to understand. 
  6. Pair with QAs. Perform testing. You will understand the domain, and the interdependencies very fast.
  7. Ask questions, write notes, and volunteer to take sessions on project topics / technologies. Nothing teaches us faster, than the pressure to not look foolish. 
  8. Try and understand the reasoning behind why features are being built, because 60% of the features in all software products are built for the same purpose — each using a different technology or tool. For instance, every corporate website has Search, Page Analytics, Splunk Style Logging, Meta-Tagging for SEO, JS/CSS optimizations for performance, Device & Browser Detection and Cookie Manipulations for Personalization, Responsive design, REST based integration, SSL, etc.  
  9. Follow a user journey in code — to understand all the layers involved, and how they interact. A decent codebase, usually has a few patterns that are repetitively used. 
  10. Use a good IDE for navigating code. They will help you understand code paths, callers, implementors, etc very quickly. I use IntelliJ (Cmd+Alt+F7), and Sublime. 
  11. Read the unit test to understand the class. Of course, I work at ThoughtWorks, and therefore have the luxury of seeing self documenting unit tests. And this means, I am indebted to ensure that readers of my code also get a good unit test.
  12. Hunt out for project documentation — especially those which pertain to large scale features, architecture, etc — as they summarize information quite well.
  13. Sign up for devops tasks. They will help you understand the interdependencies between systems very quickly.
  14. Find out who's who on the project (customer side), so that you know whom to reach out to, for what insight. 
  15. Have patience. It takes a minimum of 3 months to “feel” effective. And then another 3 to be “one-with-the-project”.


I know. It’s not a shortcut. It’s a path.


Saturday, July 26, 2014

Difference between sorted, sortWith and sortBy in Scala

Scala collections provide you three options for sorting: sorted( ), sortWith( ) and sortBy( ). Here is a simplified explanation:

sorted
Will sort the list using the natural ordering (based on the implicit Ordering passed)

sortBy (an attribute)
Sort by a given attribute using the attribute's type.
e.g. given a list of Person objects, if you want to sort them in ascending order of their age (which is an Int), you could simply say: personList.sortBy(_.age)

sortWith (a function)
Takes a comparator function. Useful when you want to specify a custom sorting logic. 
e.g. if you want to sort by age descending, you could write this as: 

personList.sortWith{(leftE,rightE) => 
     leftE.age > rightE.age
}

Or, more simply: personList.sortWith(_.age > _.age)

Checkout this gist for a full example: 
https://gist.github.com/gsluthra/80555ed4af24bea244b5



Sunday, May 4, 2014

Using Puppet to open port 80 through the iptables command

Puppet provides an add-on module called firewall to manage firewall configuration on your system. I tried it out, and ended up locking myself out of my Vagrant box. All I needed to do was open port 80 on my VM, and the steps mentioned in the Puppet Firewall module setup page seemed like an overkill for something so simple.

So, I decided to understand the iptables command better. This writeup on centos website is an excellent introduction to understanding iptables. Armed with this knowledge, I realized all I needed was:
  1. Make a single rule entry on my Centos VM for allowing incoming traffic on port 80.
  2. Save the state of the iptables rule, so that on restart of the iptables service, this new rule is not lost.
Note: I am working on puppet v3.2.3 and Cento v6.4 Minimal Version.

If you google, most places you will find this command for opening port 80 (didn't work!):

sudo iptables -A INPUT -m state --state NEW -p tcp --dport 80 -j ACCEPT

What this command essentially says is append (-A) a new rule to INPUT traffic chain, where for NEW connections of type tcp, and destination port (-dport) 80, perform ACCEPT connections.

This command did not actually open the port for me because this command "APPENDS" the rule right at the end of the iptable chain. By default, Centos already came with a rule:

-A INPUT -j REJECT --reject-with icmp-host-prohibited

The line I added, got appended AFTER this rule. Meaning, all requests were getting blocked anyways, and hence my rule to allow incoming traffic to port 80 was never evaluated.

Therefore, what we need to do is to insert our rule before the REJECT rule. To do that, we use the -I (insert) switch, instead of the -A (Append) switch. The -I switch needs to know the line number of the rule to insert at. To see the line numbers of various, use the following command:

ipTables with Line Number (Notice Line 5 Has the Reject Rule)

Using the "sudo iptables -L -n -v --line-numbers" you can see that by default line number 5 has the REJECT rule for all traffic.

Hence the command that we need to allow incoming traffic to port 80 is (works!):

sudo iptables -I INPUT 5 -m state --state NEW -p tcp --dport 80 -j ACCEPT

Now, our rules gets inserted at line 5, and everything works. Based on this knowledge, our puppet script now looks like this (using two execs):



The first exec fires the command only if it does not detect a port 80 entry (using the unless attribute). The unless attribute tests that a grep on iptables-save command contains port 80 entry. The iptables-save command allows us to see the current iptables rule configuration in a parseable format.

The second exec fires the service iptables save  command to save the configuration to disk. This command needs to be fired only if the first exec actually makes an entry. Hence we specify notify attribute in exec and also mark this exec with refreshonly => true so that it is executed only as part of the notify process, and should not be executed otherwise.

That's it! You should be good to go now! You can do the same thing if you also want to make an entry for port 443 (SSL) or anything similar.

Sunday, April 13, 2014

Reading Environment Variables in Puppet

Puppet uses a tool called Facter to discover facts about the system it is going to provision. Some examples of these facts are: $operatingsystem, $hostname, $processorcount, etc. These facts are available as variables in puppet to use in manifests. You can see the full list here: Factor 2.0 Core Facts.

If you want an environment variable, or a bash script parameter to be available to puppet, you need to make it available as a Facter fact. You can do that by simply setting a variable whose name is prefixed with the "FACTER_" string.

So for instance, if you wanted to make a variable called say $my_module_name available within puppet, and wanted its value to be equal to environment variable PRODUCT_MODULE_NAME, you could do the following in a bash script that invokes puppet:

export FACTER_my_module_name=$PRODUCT_MODULE_NAME

Now, $my_module_name variable will be available within your puppet manifests.


Setting a Default if Environment Variable is not set


To take this further, if you wanted your puppet variable to have a default value, if no environment variable was set, you could specify it in puppet manifest (.pp) file like this:

$module_name = $my_module_name ? {
      undef => "Admin_Module",
      default               => $my_module_name 
}

Note that the above code uses a different variable name called "module_name", whose value will be set to "Admin_Module" if $my_module_name is un-defined, else it will be set to value of "my_module_name" variable.
Now you can use $module_name in all your puppet manifests (instead of using $my_module_name).

To see a sample code where I did this in Bahmni Hospital Management System provisioning scripts, check out this Github commit: Example.

Tip: If you wish to debug what facts are being set in your system, you can use this snippet to print all facts to a file: Printing all puppet facts.

Tip: If your puppet script is running with "sudo", then the environment variables set in current environment won't be passed to the sudo environment. If you want that, then use "-E" switch with sudo to pass environment variables forward.


Saturday, March 1, 2014

Autojump: A neat command line utility that saves me tons of time

Autojump is a really neat utility that complements the cd command very nicely. For instance if I have a folder called "my_project", I can just say "j project", and the auto jump command will directly jump to this directory (wherever it is in my filesystem).

How does it work?

Autojump tracks how much time you spend in your command line on a specific folder. And builds a weight chart. So, if you have spent enough some time in your my_project directory on command line, and you say "j project", it will check its registry and jump to the highest match. And, the best thing is you don't even need to specify the full name. Partial names work!

Autojump in action

Install zsh. And install autojump. You will see a super productivity increase on command line.

Platforms Supported: Linux and Mac.

To install on Mac:
brew install autojump
And then copy the single line printed by brew install into your .bashrc or .zshrc file. After that go to command line, cd into a directory. Then cd to some other directory. And type "j ". You will see it work! 

Thanks to my colleague Ankit Dhingra for introducing me to auto jump. Goodbye to cd aliases.

Related Links:
https://github.com/robbyrussell/oh-my-zsh
https://github.com/joelthelion/autojump




Saturday, February 1, 2014

Displaying a Bootstrap Modal in Rails with an AJAX call

I am in the process of building a simple Learning Management Solution. I wanted to display a Bootstrap modal pop-up on click of the Add Capsules button. The idea was that on click of this button, an AJAX call should be made to the server, with the current Learning Path ID, so that it can query the DB, and find the potential capsules to be returned for the learning path. I would return an HTML snippet as response, which I wanted to insert into the body of the modal.

See screenshot below of how it looks:

Bootstrap Modal Pop up on click of Add Capsules button (with Ajax response body)












Check out this commit on Github to see the work that was needed: Bootstrap Modal Commit. The comment on the Github explains the details. It was actually quite easy once you know how.

I believe instead of sending an HTML snippet from the backend, a better practice is to send JSON data, and then construct the HTML at the browser via javascript. I guess, I could do that too, but this was really simple to do. I might modify it to return JSON some time in the recent future.

Reference:

Friday, January 10, 2014

Using Capybara and RSpec assertions in Page Objects

On my Rails project, I am using RSpec and Capybara to write functional tests.  I did not want to specify any HTML elements in my Capybara feature files, since that makes the feature files brittle to HTML / CSS changes. It also violates the DRY principle and basic code hygiene.

So, I decided to refactor out my HTML centric Capybara code into separate Page Objects. If you are unfamiliar with Page Objects, then read the following:



The Page Object pattern for encapsulating HTML centric DSL is a common pattern followed while writing UI level functional tests in ThoughtWorks.

The problem I was facing when I refactored my code into Page objects was that I was unable to use the RSpec 'expect' syntax in Page Objects. Turns out all I had to do in my page objects was:

include RSpec::Matchers

Here is the full code from my project on Github.
https://github.com/gsluthra/dakshina/tree/master/spec/features

The appropriate feature files and page objects in a GIST:
https://gist.github.com/gsluthra/8356015