Posts Tagged ‘programming’

Scrapy, an opensource screen scraping framework in Python

Monday, June 30th, 2008

Scrapy, an opensource screen scraping framework in Python

Look!, with this framework, you could easily crawl the web and take automatically that bit that you ever wanted! …  Better than this, you can take those similar bits from that bunch of cool websites you have bookmarked a long time ago and said “some day they will be useful”.  The day have arrived, you can take your hands on Python code and start programming your application to take all the stuff you want from the web. You will love standards more and more. Ahh!, and don’t forget to join us and help, the spirit of open source software is present once again, have fun scraper! http://scrapy.org/

Join strings recursively

Sunday, January 20th, 2008

I didn’t want to search for it, so I made mine:

  • python:
def join(l):
    """ Joins pieces with a connector recursively """
    if type(l) is list:
        if len(l) < 3:
            exit("ERROR: there aren't any pieces to join")
        c0 = join(l[0])
        c2 = join(l[2:]) if len(l) > 3 else join(l[2])
        if all([c0,c2]):
            return “%s%s%s” % (c0, l[1], c2)
        else:
            return c0 or c2
    else:
        return l
  • php:
/**
 * Function to construct a "clean" text by passing
 * an array of words and its connectors.
 */
function join_clean($words){
    if(is_array($words)){
        if(count($words) < 3)
            die("ERROR: there aren't any pieces to join");
        $c0 = join_clean($words[0]);
        if(count($words) > 3)
            $c2 = join_clean(array_slice($words, 2));
        else
            $c2 = join_clean($words[2]);
        if($c0 && $c2)
            return $c0.$words[1].$c2;
        elseif($c0)
            return $c0;
        else
            return $c2;
    }else
        return $words;
}