RORLAMP Development

N-choose-R

nilesh — Sun, 29 Jan 2012 14:51:35 +0000

Simple Solution for creating Permutations

While working on a research project, i came across a simple task of creating sets of permutation for a given objects. What i thought simple task become tedious after goggling it. I didn’t find any simple solutions for creating a permutations. So i create his one for those who wants simple solution for creating permutations.

How to solve complicated problems of creating permutation.

Creating permutation of object such as set of colors, basically the task is to find combination of colors (i.e. color addition problem)

Step 1 : create array of pointers to color objects

colors = { red, green, blue, red, …}

Step 2 : create method which take color object pointers and return a combination for selected color objects

magenta = combinations(colors, indexes), where indexes contain index of red and blue indexes = {2, 3}

Step 3 : create indexes using Permutation methods.

Creating unique distribution of samples across multiple threads (p-threads) , .e. Dividing the work across the threads such that none of the thread do the same work.


   int n;//  set of object
   int r; //  subset of chosen object

   \* for each threads do *\
   {
         pid = get_pthread_id()
         p = get_total_pthread()
         perm =  createPermutation(n , r)
         startindex = pid * (n / p)
         PermuteAfterNiteration (perm, startindex)

         /*
           use nextpermutation method which return unique index
           Performed required operations
         */

         freepermutation()
    }

 Code : 

#include ;
#include ;
#include ;

struct Permutation { 

  /* To Store current Combinations */
	int *A_i ; 

   /* To Store previous Combinations */
  int *A_i_1 ; 

  /* Total Number of Objects ( i.e. N), Object to Choose (i.e. R) */
	int n, r ; 

  /* To Store total possible Combinations */
	long int noOfPermutationCompleted ;
}; 

/* Create N-Choose-R chooser */
struct Permutation *createPermutation ( int n , int r ){ 

	int i ;
	struct Permutation * toreturn = ( struct Permutation *) malloc( sizeof ( struct Permutation ));
	toreturn -> n = n ;
	toreturn -> r = r ;
	toreturn -> noOfPermutationCompleted = 0 ;
	toreturn -> A_i = malloc ( r * sizeof ( int ));
	toreturn -> A_i_1 = malloc ( r * sizeof ( int ));
	for ( i = 0 ; i < r ; i ++){
		toreturn -> A_i [ i ] = 0 ;
		toreturn -> A_i_1 [ i ] = 0 ;
	}
	return toreturn ;
} 

/* Return next array of object indexs */
int * nextPermutation ( struct Permutation * Per ){ 

	int j ;
	size_t bytes = ( sizeof ( int ) * Per -> r - 1 ); 

	/* Per -> A_i = Per -> A_i_1; */
	memcpy ( Per -> A_i_1 , Per -> A_i , bytes ); 

	for ( j = ( Per -> r - 1 ) ; j >;= 0 ; j --){
		Per -> A_i [ j ] += 1 ;
		if ( Per -> A_i [ j ] < Per -> n )
			break ;
		else
			Per -> A_i [ j ] = 0 ;
	} 

	/* Per -> A_i = Per -> A_i_1; */
	memcpy ( Per -> A_i_1 , Per -> A_i , bytes ); 

	Per -> noOfPermutationCompleted += 1 ; 

	return Per -> A_i ;
} 

/* It initialise N-Choose-R chooser to start after Nth Iteration */
void PermuteAfterNiteration ( struct Permutation * Per , long int Niteration ){ 

	 while ( Niteration != 0 ){
	 	nextPermutation ( Per );
	 	Niteration --;
	 }
} 

/* Return total Permutation, for N-choose-R chooser */
long int TotalPermutations ( struct Permutation * Per ){ 

	long int ret = 1 ;
	int i = 0 ; 

	for ( i = 0 ; i < Per -> r ; i ++)
		ret *= Per -> n ; 

	return ret ;
} 

/* Free Memory */
void free_Permutation ( struct Permutation * Per ){ 

	free ( Per -> A_i );
	free ( Per -> A_i_1 );
	free ( Per );
}
/*
Demo Program

int main(){

	int j, *permute, N = 4, R = 2, Nth = 10;
	long int tot, i;

	struct Permutation *Per = createPermutation(N, R);

	tot = TotalPermutations(Per);

	printf("Total Permutation %ld \n\n", tot);
	printf("List all Permutations \n");
	for(i = 0 ; i < tot ; i++){
		permute = nextPermutation(Per);
		printf("permutation %ld : ",i + 1);
		for(j = 0 ; j < R ; j++)
			printf("%d ", permute[j]);
		printf("\n");
	}

	printf("\nAfer Using nth Iterator \n");
	PermuteAfterNiteration(Per, Nth);
  	printf("List all Permutations \n");
	for(i = 0 ; i < tot - Nth ; i++){
		permute = nextPermutation(Per);
		printf("permutation %ld : ",i + 1);
		for(j = 0 ; j < R ; j++)
			printf("%d ", permute[j]);
		printf("\n");
	}

	printf("Free Memory \n");
  	free_Permutation(Per);

	return 0;
}

Output :
  Total Permutation 16

  List all Permutations
  permutation 1 : 0 1
  permutation 2 : 0 2
  permutation 3 : 0 3
  permutation 4 : 1 0
  permutation 5 : 1 1
  permutation 6 : 1 2
  permutation 7 : 1 3
  permutation 8 : 2 0
  permutation 9 : 2 1
  permutation 10 : 2 2
  permutation 11 : 2 3
  permutation 12 : 3 0
  permutation 13 : 3 1
  permutation 14 : 3 2
  permutation 15 : 3 3
  permutation 16 : 0 0

  Afer Using nth Iterator
  List all Permutations
  permutation 1 : 2 3
  permutation 2 : 3 0
  permutation 3 : 3 1
  permutation 4 : 3 2
  permutation 5 : 3 3
  permutation 6 : 0 0

  Free Memory

*/

PubSub Chat Using HTML5 Web Socket and em-websocket

Amar Daxini — Thu, 29 Dec 2011 02:10:29 +0000

It’s 5 in the morning and i am tired and its a possibility that i may be writing a long post, Last night i asked my self what is it that i have learned in the las t couple of months and out of nothingness i rememberd a VIDEO CHAT APPLICATION which my friend Viren and I worked on. I thought it might be useful for some of you.

When we started, we didnt have enough knowledge where to begin with and had time constraints.We came across solutions like Red5,ErlyVideo.

Now out problem was that out client had to be in flash or java applet, we we tried lots of things and the mother solutions of all problems was yet released, HTML5.

When we started, we didnt have enough knowledge where to begin with and had time constraints to meet. We came across solutions like Red5,ErlyVideo. Now our problem was that our client had to be in flash or java applet. So we tried working around many solutions. HTML5 was months away from its release.

we are looking into other HTML5 Features and we get awesome HTML5 web socket support so we thought let’s get some hands on it.

So we think let’s build pubsub chat application, after Googling we got couple of solution.but we want to build own i have choice between nodejs and EventMachine.we have choose EM to quick start.

So we are ready,
We go through HTML5 Web Socket API,and tutorial so basically it’s a technique for two-way communication over one (TCP) socket, a type of PUSH technology.
So it has mainly three events

onopen: When a socket has opened
onmessage: When a message has been received
onclose: When a socket has been closed

More info regarding HTML5 Web Socket you can visit Nettuts, HTML5Demo.
lya Grigorik has written awesome library,and article on it so i don’t go in much detail.

Steps we are following

When User is connected onopen event is fired and we create one default connection and add to pool of channels,it’s used for mainly notification purpose

When User A wants to chat with other User B we pass user id of both user and we create channel on that.
So basically User A is subscribe to individual channel for simplicity i have taken user id for creating uniq channel.I can use session or other stuff currently i am not worry about security.
When User A start sending messages to User B,it fires onmessage event,it first populate into EM Queue,right now i don’t want to use any dependency,this can be done with easily with redis also.
After sending messages User B will receive notification via default channel which is created when it’s connected.
Now User B is wants to chat with User A it check whether channel is present or not if not it creates channel and subscribe onto it
If channel is present then whatever messages in en queue it will be pop out and display messages.
If suppose anyone disconnect from then onclose event is fire it checks whether default channel is present then unsubscribe from channel and remove socket from channel,and decrease uid by one
If any channel is present then unsubscribe from channel and remove socket from channel,and decrease uid by one.
Now user is again connected it will just subscribe channel.

Now I haven’t written code for removing dead channel it will be done easily.Code or Idea is may or may not fully applicable,there are various solutions are available and respective opensource library is also present.

Now same concept can be easily applicabale redis pubsub also.

Code is present on github,although code is not so furnish but you can checkout and have some fun

Please feel free to share and comment on it.

At last i am awake it’s around 6.30 o’clock .

Managing daemon using daemon-kit

Kelvin Kunjukutty — Wed, 28 Dec 2011 06:30:57 +0000

Hey all,
In Previous article i have given short bio of ruby daemon using daemon kit.
In this post i am writing about how to manage daemon,how to start/stop a daemon from any directory.

We can start a daemon using following way.
Go to the directory and give the command.

./bin/mydaemon

./bin/mydaemon start

(You know the difference right; To run it in foreground or background respectively.)

But consider this possibility, what if you didn’t want to go to the directory where the daemon is located and start it.

/home/mysystem/mydaemon/bin/mydaemon

/home/mysystem/mydaemon/bin/mydaemon start

Now you might be thinking why you will want to do that, let me say such conditions does arise (when you want monit to watch over it and restart it when it goes down.)

Usually this would work with ease if you are not using a bundler, however if you are using it then this would be something that you would get to see.

/usr/lib/ruby/gems/1.8/gems/bundler-1.0.15/lib/bundler/shared_helpers.rb:22:in `default_gemfile': Could not locate Gemfile (Bundler::GemfileNotFound)

This happens because bundler looks for the Gemfile in the directory you are in and the Gemfile is not there.
So what do you do to overcome it ?
You tell the daemon where to look for the gemfile
and how do you do it?

In you daemon place the following piece of code in bin/mydaemon at the very beginning.

ENV['BUNDLE_GEMFILE'] = "Path/to/your/daemon/Gemfile"

Now this is all fine and you have got your daemon working…. BUT what if others would also want to work with your daemon in there system.

This won’t work as the location given previously would not be the same as theirs.
Don’t worry I have a solution for that too.

Instead of

ENV['BUNDLE_GEMFILE'] = "Path/to/your/daemon/Gemfile"

put it as

ENV['BUNDLE_GEMFILE'] = File.join(File.dirname(__FILE__),'..','Gemfile')

This would help in anybody starting the daemon from any directory if required or starting the daemon by going to the directory and running it.

And thus you have your daemon running from your daemon root and other different location.
Hope this helps you at some point of time.

Understanding Daemonkit

Kelvin Kunjukutty — Thu, 22 Dec 2011 08:28:51 +0000

Hey all,

Many of us knows what daemons are in linux. These are processes or programs that run in background with little or no user intervention.

Now we can write daemons in ruby too. And for doing so we can use Daemon-Kit(https://github.com/kennethkalmer/daemon-kit).

Now the daemon can do the task continuously or do it at particular interval of time or do it depending on a particular event.

Daemon-Kit provides a skeleton to write these different types of daemons with the help of differnt types of generators.(XMPP bot,AMQP client,Nanite agent,Cron-style,ruote remote participants). Of these XMPP and AMQP are event based and amqp is specifically based on queuing system, and Nanite agent gives a nanite structured daemon while Cron-style is used to write daemons that perform certain task at particular interval amount of time.

First install the gem daemon-kit

gem install daemon-kit

For viewing the help of daemonkit you can give the command,

daemon-kit -h

Now to build a simple daemon which would use a default generator you can use,

daemon-kit mydaemon

For building a daemon with a generator you can use,

daemon-kit mydaemon -i amqp

daemon-kit mydaemon -i cron

Now for starting the daemon first you go into the daemon directory and then give the command,

cd mydaemon

./bin/mydaemon

which would start the daemon in foreground which is used for testing if everhting is working.The daemon would die as you close the terminal.

To purely daemonize it you can use

./bin/mydaemon start

This would start the daemon in daemonized form and would continue to run in background till any untoward exception occurs.

To stop the daemon running in background you can use the commnad

./bin/mydaemon stop

Now the basic tree structure of a daemon would be something similar to this.

my_cron_daemon

├── bin

│ └── my_cron_daemon

├── config

│ ├── arguments.rb

│ ├── boot.rb

│ ├── environment.rb

│ ├── environments

│ │ ├── development.rb

│ │ ├── production.rb

│ │ └── test.rb

│ ├── post-daemonize

│ │ └── readme

│ └── pre-daemonize

│ ├── cron.rb

│ ├── readme

│ └── safely.rb

├── Gemfile

├── lib

│ └── my_cron_daemon.rb

├── libexec

│ └── my_cron_daemon-daemon.rb

├── log

├── Rakefile

├── README

├── script

│ ├── console

│ ├── destroy

│ └── generate

├── spec

│ ├── my_cron_daemon_spec.rb

│ ├── spec_helper.rb

│ └── spec.opts

├── tasks

│ └── rspec.rake

├── tmp

└── vendor

The pre-daemonize folder consists of all the files that are required before the daemon starts. The cron.rb can be used to require all the gems that are needed for the daemon.

The my_cron_daemon.rb in lib folder is automatically loaded when the daemon starts. Additonal ruby files can be created in the lib folder and can be required from the cron.rb in pre-daemonize for utilization.This can be done as,

excluded_files = [File.join(DaemonKit.root, 'lib', "my_cron_daemon.rb")]

(Dir.glob(File.join(DaemonKit.root, "lib", "*.rb")) - excluded_files.flatten).each{|f| require f}

This is done because my_cron_daemon.rb in lib is automatically loaded and requiring it multiple times might cause issues.

The post-daemonize folder is used to place those files that needs to be loaded after the daemon starts.

Similarly a yml file(e.g config.yml) which holds all the configuration setting can be created in the config and loaded into the daemon using,

CONFIG = DaemonKit::Config.load("config.yml")

Here CONFIG can be used anywhere in the daemon later on to access the required configuration.

Now you can also add ruby files that connect to a particular database and query data by putting those in lib. These files would have a structure similar to the ruby files in models of rails application.

As for the database connection you can put these pieces of code in cron.rb in pre-daemonize which establishes the databse connection,

ActiveRecord::Base.configurations = CONFIG[:database_configuration]

ActiveRecord::Base.establish_connection(CONFIG[:database_configuration])

where “database_configuration” is placed in the config.yml

Now we can also have the daemon as a request-response serving program similar to a rack-server, This can be done by using eventmachine.

Daemons are a difficult to debug. Because they can just cause your program to fail when a request comes in. Normally we write a code in a normal Ruby program and run it to test all the functionallity and then put it within the daemon. This helps in ensuring that your logic is proper. For further debugging I use a lot of puts statements when coding later switching to DaemonKit.logger.info or .debug to dump the statements to the log.

DaemonKit.logger.info "I am the DaemonKit logger info"

Here is a link to a basic daemon which is based on default,cron generator.

https://github.com/kelvink/My-Basic-Daemon

https://github.com/kelvink/My-Cron-Daemon

Also here is a link for the daemon to work as a rack server using evenmachine.

https://github.com/kelvink/My-EM-Daemon

Thus we can have a ruby daemon which perform specific task as required and when required.

Hey all,

Many of us knows what daemons are in linux. These are processes or programs that run in background with little or no user intervention.

Now we can write daemons in ruby too. And for doing so we can use Daemon-Kit.

Now the daemon can do the task continuously or do it at particular interval of time or do it depending on a particular event.

Daemon-Kit provides a skeleton to write these different types of daemons with the help of differnt types of generators.(XMPP bot,AMQP client,Nanite agent,Cron-style,ruote remote participants). Of these XMPP and AMQP are event based and amqp is specifically based on queuing system, and Nanite agent gives a nanite structured daemon while Cron-style is used to write daemons that perform certain task at particular interval amount of time.

First install the gem daemon-kit

gem install daemon-kit

For viewing the help of daemonkit you can give the command,

daemon-kit -h

Now to build a simple daemon which would use a default generator you can use,

daemon-kit mydaemon

For building a daemon with a generator you can use,

daemon-kit mydaemon -i amqp

daemon-kit mydaemon -i cron

Now for starting the daemon first you go into the daemon directory and then give the command,

cd mydaemon

./bin/mydaemon

which would start the daemon in foreground which is used for testing if everything is working.The daemon would die as you close the terminal.

To purely daemonize it you can use

./bin/mydaemon start

This would start the daemon in daemonized form and would continue to run in background till any untoward exception occurs.

To stop the daemon running in background you can use the command

./bin/mydaemon stop

Now the basic tree structure of a daemon would be something similar to this.


my_cron_daemon

├── bin

│   └── my_cron_daemon

├── config

│   ├── arguments.rb

│   ├── boot.rb

│   ├── environment.rb

│   ├── environments

│   │   ├── development.rb

│   │   ├── production.rb

│   │   └── test.rb

│   ├── post-daemonize

│   │   └── readme

│   └── pre-daemonize

│       ├── cron.rb

│       ├── readme

│       └── safely.rb

├── Gemfile

├── lib

│   └── my_cron_daemon.rb

├── libexec

│   └── my_cron_daemon-daemon.rb

├── log

├── Rakefile

├── README

├── script

│   ├── console

│   ├── destroy

│   └── generate

├── spec

│   ├── my_cron_daemon_spec.rb

│   ├── spec_helper.rb

│   └── spec.opts

├── tasks

│   └── rspec.rake

├── tmp

└── vendor

The pre-daemonize folder consists of all the files that are required before the daemon starts. The cron.rb can be used to require all the gems that are needed for the daemon.

The my_cron_daemon.rb in lib folder is automatically loaded when the daemon starts. Additonal ruby files can be created in the lib folder and can be required from the cron.rb in pre-daemonize for utilization.This can be done as,

excluded_files = [File.join(DaemonKit.root, 'lib', "my_cron_daemon.rb")]

(Dir.glob(File.join(DaemonKit.root, "lib", "*.rb")) - excluded_files.flatten).each{|f| require f}

This is done because my_cron_daemon.rb in lib is automatically loaded and requiring it multiple times might cause issues.

The post-daemonize folder is used to place those files that needs to be loaded after the daemon starts.

Similarly a yml file(e.g config.yml) which holds all the configuration setting can be created in the config and loaded into the daemon using,

CONFIG = DaemonKit::Config.load("config.yml")

Here CONFIG can be used anywhere in the daemon later on to access the required configuration.

As for the database connection you can put these pieces of code in cron.rb in pre-daemonize which establishes the databse connection,

ActiveRecord::Base.configurations = CONFIG[:database_configuration]

ActiveRecord::Base.establish_connection(CONFIG[:database_configuration])

where “database_configuration” is placed in the config.yml

Now we can also have the daemon as a request-response serving program similar to a rack-server, This can be done by using eventmachine.

DaemonKit.logger.info "I am the DaemonKit logger info"

Here are links to basic daemon which is based on default,cron generator and rack server using eventmachine.

https://github.com/kelvink/My-Basic-Daemon,

https://github.com/kelvink/My-Cron-Daemon,

https://github.com/kelvink/My-EM-Daemo

Thus we can have a ruby daemon which perform specific task as required and when required.

acts_as_flying_saucer on Rails 3.1 and Heroku

Amar Daxini — Mon, 24 Oct 2011 10:03:36 +0000

Just quick post how to use acts_as_flying_saucer with rails 3.1 and Heroku.

I was testing acts_as_flying_saucer with rails 3.1.It is working fine so far.

but when i have added external style sheet it was hanging on dev mode.

and later on i figure out it is external style sheet causing an issue.

So i precomplied css and then try to generate pdf it is working.

Now it is time to test on heroku.

I am using cedar stack and ruby 1.9.2 after precomplied step pdf.

By default pdf is generated on system /tmp directory.

In heroku we can set application tmp path to generate pdf i.e ./tmp

I was testing acts_as_flying_saucer with rails 3.1.It is working fine so far,but when i have added external style sheet it was hanging on dev mode rails is hanging

After litte bit of googling i figure out it is external style sheet causing an issue because of asset pipeline and thread issue for serving static file mainly in dev mode.

So i simply pre complied css or we can manually (actual full path) or manipulate asset path .after this PDF is generated properly.

Now it is time to test on Heroku.I am using cedar stack and ruby 1.9.2 after pre complied step.

Now point tmp directory to application tmp directory.

ActsAsFlyingSaucer::Config.options = {:tmp_path => "./tmp"}

Walk through of new features of acts_as_flying_saucer XHTML to PDF

Amar Daxini — Mon, 24 Oct 2011 09:24:37 +0000

It’s about more than 6 months i haven’t written any post.

Why ?

Recently join new organiztion,

Can’t get time because we are working almost more than 10 to 12 hr day with messy code.

So we are spending more time to refactoring and optimising code,and scaling application also.

Now i started making my time to writing.

In my current organization we are generating various Pdf report.

We are using prawn but now we are switching to acts_as_flying_saucer.

So there are couple of challenges i have faced

1) XHTML code is not proper

2) Special character in html

Solution

Cleanup Html code

I am using tidy library clean up html code.

By passing :clean=>true it will cleanup html befor generating pdf.

3) Send PDF as email

Solution

Generate PDF on server and do some processing like attached to mail.

:send_attachment => true it will generate pdf and return path of pdf file then do some processing.

So couple of new feature is added to acts_as_flying_saucer

1) Clean up html by passing :clean=>true

2) Generate pdf on server (locally) by passing :send_attachment=>true

3) acts_as_flying_saucer is also working with rails 3.1 (precomplie asset) and heroku.

It’s about more than 6 months i haven’t written any post Why ?

because recently join new organization, and can’t get much more time , we are working almost more than 10 to 12 hr day with messy code.So we are spending more time to refactoring and optimizing code,and scaling application.

In my current organization we are generating various PDF report.We are using prawn but now we are switching to acts_as_flying_saucer.

So there are couple of challenges i have faced

XHTML code is not proper
Special character in html

Solution: Cleanup Html code

I am using tidy library clean up html code. By passing :clean => true it will cleanup html before generating PDF.

3. Send PDF as email or do some processing

Solution: Generate PDF on server and do some processing like attached to mail.

:send_to_client =>false It will generate pdf and return path of pdf file then do some processing.

4. Generate PDF from String,File,URL

:url – URl/String/File Path Now by passing URL or String or File Path PDF can be generated.

So couple of new feature is added to acts_as_flying_saucer is as follows

1) Clean up html by passing :clean=>true

2) Generate pdf on server (locally) by passing :send_to_client=> false

3) Fixed issue of non inline CSS issue.

4) Generate PDF from File,String,URL.

5) More info about acts_as_flying_saucer you can visit wiki

6) PDF can be generated from Rails,Sinatra,Ruby.

Custom Pre-Processor In Rails 3.1

viren — Tue, 30 Aug 2011 09:55:15 +0000

Well long break,Let me guess It over 3 month since I wrote my last post. I’ve been kind of busy lately not manage to dedicate much time to thing that I do.

Rails 3.1.0 is been in picture for a quite a while a now and major feature that been included in Rails 3.1.0 is asset pipeline and pre-processor.

For those is who aren’t aware of Rails 3.1.0 asset pipeline and pre-processor there is a very good article on rails guide and recently Ryan Bates introduce a screencast on Rails 3.1.0 asset pipeline.

I’m not here to talk about asset pipeline you can get it all if you just googled it.

So what I’m here upto then ?

Well ever since I was made aware of what asset pipeline and pre-processor more importantly pre-processor ,it always trilled me of using a custom pre-processor .

As you all know that Rails 3.1.0.rc1 is bundle with coffee-script and sass support. In your asset folder you may find js and css file that look something like this “*.js.coffee” , “*.css.scss“. Now you can add a chain of pre-processor that just look like this “*.js.coffee.erb” , “*.css.scss.erb”

The post is all about chaining your own pre-processor just as above something like “*.js.coffee.mustache”

Mustache (Mustache is taken just for example here you can use your own) because I been in love with it ever since I explored it . So how to accomplish this .

Stay with me as I will show how ?

Firstly you need to be aware of Sprockets and Tilt library

First the code

config/initializers/mustache_template.rb

require "mustache"
require "tilt"

class MustacheTemplate < Tilt::Template
  def prepare; end

  def evaluate(context, locals, &block)
    @data = ::Mustache.render @data ,:mustache_variable => "World" ,:name => "Viren Negi"
  end

end

Rails.application.assets.register_preprocessor 'application/javascript', MustacheTemplate

That all it take to define a custom preprocessor Before I explain the above code let me show what exactly my preprocessed file look like .

app/assets/javascripts/user.js.coffee.mustache

# Here is the custom mustache and coffee-script code

a =  "Hello {{ mustache_variable }} from {{ name }}";

if a
  alert a
else
  alert "Sorry LOL"
  alert "The co-processor is not working"

well as you would have guess I received and alert prompt on UI stating “Hello World from Viren Negi”

Well that quite neat .

Let me describe what happening under the hood.

Rails.application.assets is Sprocket::Environment instance as you would guess register_preprocessor is Sprocket method to register the preproccessor .

a closer look at the Sprocket ” register_preprocessor “ method

sprockets/lib/sprockets/processing.rb

 def register_preprocessor(mime_type, klass, &block)
      expire_index!

      if block_given?
        name  = klass.to_s
        klass = Class.new(Processor) do
          @name      = name
          @processor = block
        end
      end

      @preprocessors[mime_type].push(klass)
    end

As you guess till now that the method except 2 argument and block (if given).

For all those dump guys like me who still haven’t figure it out.
What it basically does is “it push the class to mime_type” in our case (mime_type = “application/js” and klass = MustacheTemplate)

what it internally does when processing the template is some Tilt magic.

The ruby code


def prepare ;   end

def evaluate(context,locals,&block)

...

...

end

defined above are basically the Tilt method which is overridden in MustacheTemplate to process the template .

That it that how you add your own custom pre-processor .

Hope that it help you in same way or other.

Thanks

Viren Negi

Passing arrays using Net-http in Ruby

Kelvin Kunjukutty — Tue, 23 Aug 2011 06:04:55 +0000

Hi all,

We needed to use net/http of ruby for sending parameters between two modules.

We decided to use the get and the post request of http as follows:

url = URI.parse(”http://SERVER:PORT”)

http = Net::HTTP.new(url.host, url.port)

request = Net::HTTP::Get.new(”/method_name?parameter=a”)

response = http.request(request)

In the above get request the parameter “a” is taken to the “http://SERVER:PORT/method_name” and data related to it is queried.

url = URI.parse(”http://SERVER:PORT”)

http = Net::HTTP.new(url.host, url.port)

request = Net::HTTP::Post.new(”/method_name”)

request.set_form_data({”parameter1″ => “a”, “parameter2″ => “b”, “parameter3″ => “c” })

response = http.request(request)

In the above post request three parameters are taken parameter1, parameter2 and parameter3 with the value a,b and c respectively to the “http://SERVER:PORT/method_name” for posting.

When you need to pass a array in the parameters for posting. you can specify it as:

request.set_form_data({”parameter1″ => “a”, “parameter2″ => ["1", "2", "3"], “parameter3″ => “c” })

Here the parameter2 takes a array with value 1,2,3.

Now the fun begins,

If you are using ruby 1.9.2 this works fine parameter2 is passed as a array.(parameter2 = ["1", "2", "3"])

However if you are uing ruby 1.8.7 this doesnot work, parameter2 appends all the values of the array to form a string and passes it.(parameter2 = “123″)

You have to override the set_form_data method for it to work and pass parameter as a array.

It can be done as below. You an place this in a file and let the file be included while loading the app.

module Net

module HTTPHeader

def set_form_data(request, params, sep = ‘&’)

request.body = params.map {|k,v|

if v.instance_of?(Array)

v.map {|e| “#{urlencode(k.to_s)}=#{urlencode(e.to_s)}”}.join(sep)

else

“#{urlencode(k.to_s)}=#{urlencode(v.to_s)}”

end

}.join(sep)

request.content_type = ‘application/x-www-form-urlencoded’

end

alias form_data= set_form_data

def urlencode(str)

str.gsub(/[^a-zA-Z0-9_\.\-]/n) {|s| sprintf(’%%%02x’, s[0]) }

end

And then use set_form data as follows:

request.set_form_data(request, {”parameter1″ => “a”, “parameter2″ => ["1", "2", "3"], “parameter3″ => “c” })

Thus we can pass a array in parameters while use net/http of ruby 1.8.7.

References : http://blog.assimov.net/post/653645115/post-put-arrays-with-ruby-net-http-set-form-data

In one of our projects, we needed to use net/http of ruby for sending parameters between two modules which resided on two different servers.

We decided to use the get and the post request of http as follows:

url = URI.parse("http://SERVER:PORT")

http = Net::HTTP.new(url.host, url.port)

request = Net::HTTP::Get.new("/method_name?parameter=a")

response = http.request(request)

In the above get request the parameter “a” is send to the “http://SERVER:PORT/method_name” and data related to it is queried.

url = URI.parse("http://SERVER:PORT")

http = Net::HTTP.new(url.host, url.port)

request = Net::HTTP::Post.new("/method_name")

request.set_form_data({"parameter1" => "a", "parameter2" => "b", "parameter3" => "c" })

response = http.request(request)

In the above post request three parameters are taken parameter1, parameter2 and parameter3 with the value a,b and c respectively to the “http://SERVER:PORT/method_name” for posting.

When you need to pass a array in the parameters for posting. you can specify it as:

request.set_form_data({"parameter1" => "a", "parameter2" => ["1", "2", "3"], "parameter3" => "c" })

Here the parameter2 takes a array with value 1,2,3.

Now the fun begins,

If you are using ruby 1.9.2 this works fine parameter2 is passed as a array.(parameter2 = ["1", "2", "3"])

However if you are uing ruby 1.8.7 this doesnot work, parameter2 appends all the values of the array to form a string and passes it.(parameter2 = “123″)

You have to override the set_form_data method for it to work and pass parameter as a array.

It can be done as below. You an place this in a file and let the file be included while loading the app.

module Net

module HTTPHeader

def set_form_data(request, params, sep = '&')

request.body = params.map {|k,v|

if v.instance_of?(Array)

v.map {|e| "#{urlencode(k.to_s)}=#{urlencode(e.to_s)}"}.join(sep)

else

"#{urlencode(k.to_s)}=#{urlencode(v.to_s)}"

end

}.join(sep)

request.content_type = 'application/x-www-form-urlencoded'

end

alias form_data= set_form_data

def urlencode(str)

str.gsub(/[^a-zA-Z0-9_\.\-]/n) {|s| sprintf('%%%02x', s[0]) }

end

end

end

And then use set_form data as follows:

 request.set_form_data(request, {"parameter1" => "a", "parameter2" => ["1", "2", "3"], "parameter3" => "c" })

Thus we can pass a array in parameters while use net/http of ruby 1.8.7.

If any other solutions please feel free to share.

References : http://blog.assimov.net/post/653645115/post-put-arrays-with-ruby-net-http-set-form-data

Word Search Using grep and AWK

Kelvin Kunjukutty — Thu, 23 Jun 2011 12:23:43 +0000

In one of our project we needed to search for multiple words from a file and return the found words.
These files would be large and there might be large number of words to search for also.

Ruby has a good support for regexp, however it would be slow in doing these searches for patterns as compared to unix commands and hence we decided to look for certain unix commands for doing these searches.

We went through “grep“, “awk” and “boyermoore searching algorithm in ruby” for doing this search.
Each of these were used for searching patterns in a file.

grep and AWK were giving the results as the line(not he line number but the whole content of line) on which the matching pattern were present. Working on grep and awk further we found differnt ways of searching from files using these.

Using grep:

For searching a word from a file by grep we can say :

grep "word" filepath

e.g

grep "cool" fun.txt

For searching multiple words from a file we can use :

grep "word1\|word2" filepath

e.g

grep "cool\|hot" fun.txt

This would return any word which contains cool or hot in it.(i.e it would return even hotter as a match)

To make it a whole word compare we have to add a -w option to it.

grep -w "word1\|word2" filepath

This would return only words which contains cool or hot in it.

Also we can take the words from the list in file and compare those and search for other file.

grep -f list.txt filepath

e.g

grep -f list.txt file.txt

Here list.txt contains the list of words(each word in a new line) and the file.txt is the file to be searched in.

Now these gave the line in which the words were present as the result.

Now we wanted which words matched and not the whole line, so we added a option ” -o ” to it.

So final grep command was :

grep -o -w "word1\|word2" filepath

Using AWK:

For searching a word from a file by awk we can say:

awk '/word/' filepath

For searching the file with the whole word and not those that has word as a part of it, we can use (This is similar to using -w with grep) :

awk '/\/' filepath

For searching multiple words we can use :

awk '/\|\/' filepath

Similarly here also the result wld give the whole line as result in which the word was present, so to give only the matched words the command was :

awk '{for(i=1;i<=NF;i++){if($i~/\|\/){print $i}}}' filepath

This would split each word within the file and compare it and give the matched words out as result.

Using Boyermoore Searching Algorithm:

Also we has tried a searching algorithm written in ruby namely “boyermoore searching algorithm”

Here we had to require the library file required for it and call it as:

BoyerMoore.search("ANPANMAN", "ANP")

This would give the result as the position in which the letter has matched, which would give the result as 0 in this case.

Now by benchmarking these three types of searching methods we found that grep and awk are better.

Both of them were giving minor time difference.

So thus we know that grep and awk are one of the solution for searching patterns in a file.

Mount Remote File System Using Fuse and sshfs

Kelvin Kunjukutty — Thu, 23 Jun 2011 09:31:59 +0000

In one of the project we needed to store data in our data center i.e not place(store) it on cloud specifically. And this data needs to be available to multiple users/system. The operation on this data included creating/editing a file, copying a file from those stored data or copying to it.

After searching a few, we decided to use Fuse along with sshfs.

Fuse allows to use a remote file as a local file and allows us to make changes to it that would reflect correctly on both the remote and local files.

Mounting a remote folder on ubuntu using Fuse(Filesystem in Userspace) and sshfs(Secure SHell FileSystem):

The stable release of fuse can be downloaed from : http://fuse.sourceforge.net/

Further installation can be done by :

./configure
make
sudo make install

Further installation of sshfs can be done by :

sudo apt-get install sshfs

Later we need to load the fuse using :

sudo modprobe fuse

Further the user needs to be assigned rights for performing action on fuse:

sudo adduser  fuse

The above command is used to add the user to the fuse module.

sudo chown root:fuse /dev/fuse
sudo chmod +x /dev/fusermount

While these command mentioned above are used to change the permission access of the file.

sudo chmod +x /dev/fusermount

might give an error namely directory not found.

In this case we can use the “whereis” command to locate usermount.

whereis fusermount

which would return a result something similar to
/usr/bin/fusermount

Using this location do :

sudo chmod +x /usr/bin/fusermount

After this do a logout and login back to the system.

Now make a directory named remoteserver to mount the remote folder.

mkdir ~/remoteserver

Now to mount the remotefolder on the remoteserver use :

sshfs @:/remotepath ~/remoteserver

Here the remotepath points to the remotefolder.

After this you will be prompted for the password.

Finally you will be able to use the remotefolder as if it was a part of the local system.
The remoteserver directory will contain all of the contents of the remotefolder.
Any changes made in the remoteserver will reflect in the remotefolder too.

For unmounting the remotefolder use :

fusermount -u ~/remoteserver

Using dd command to test

dd command is used to create a file :

dd if=/dev/zero of=output.dat  bs=10M  count=10240

Here the file named output.dat is created of size 100MB.

The bs option specifies the size of the file to be created.

To see the help of “dd”
Give a

dd --help

on the console.

Using dd command I created a file in the remotefolder by creating it in the remoteserver directory.
Also we tried copying the file along the network and compared the md5sum of the files to see if the files are correct.

md5sum filepath

In this case we can do it as,

md5sum output.dat

The transfer speed across the network came as 10 Mbps. This is dependent on the speed of the network.

Mounted filesystem can be used using the above explanation.