Reportlab and Pisa

Reportlab and Pisa

PDF display or download through a web page is always an excellent idea to impress other if it's done on the fly, and Reportlab and Pisa gives you the great feature to do that.
In my case, i had a small file and using that file i have to create PDF, obviously with too many calculations and plots(Reportlab used). So, if you are using python then none other tool is as good as reportlab and Pisa.
I feel very comfortable to create HTML first and then convert it to PDF using Pisa. Here is the simple solution or you can say kind of HTML template to convert to PDF.
To understand this code, you need to go through Pisa documentation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
import cStringIO
import ho.pisa as pisa
 
def convert(data, filename, open=False):
    pdfFile = file(filename, "wb")
    pdf = pisa.CreatePDF(
       cStringIO.StringIO(data.encode("ISO-8859-1")),pdfFile)
    pdfFile.close()
    if not pdf.err:
        pisa.startViewer(filename)
 
 
if __name__=="__main__":
    HTML = """<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <html>
    <head>
        <meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1" />
        <style type="text/css" media="all">
            @page {
            margin: 0.5cm;
            margin-top:1.8cm;
            margin-bottom: 1cm;
            @frame footer {
            -pdf-frame-content: footerContent;
            bottom: 0.5cm;
            margin-left: 0.75cm;
            margin-right: 0.75cm;
            height: 1cm;
            }
            @frame header {
            -pdf-frame-content: headerContent;
            top: 0.2cm;
            margin-left: 0.75cm;
            margin-right: 0.75cm;
            height: 2cm;
            }
            }
            .footer .head
            {
                position:absolute;
                left:0;
                top:4;
                width:100%;
            }
            table {
            -pdf-keep-in-frame-mode: shrink;
            }
            h1 {
                page-break-before:always;
            }
        </style>
    </head>
    <body>
        <div id="headerContent" class="head">
        <table width="100%" height="100px" border="0" >
            <tr>
            <td wodth="25%"><img src="logo.gif" width="70px" height="80px"/>Company Name</td>
            <td align="center" width="50%"><font face="Helvetica-Bold" size="24"> Title </font></td>
            <td width="25%"><font face="Helvetica" size="4">Sub titles</font></td>
            </tr>
        </table>
        </div><br/></br>
            <h1>YOUR OTHER HTML CONTENT WILL GO HERE..........</h1><br/></br>
            <h1>YOUR OTHER HTML CONTENT WILL GO HERE..........</h1><br/></br>
            <h1>YOUR OTHER HTML CONTENT WILL GO HERE..........</h1><br/></br>
        <div id="footerContent">
              <div class="footer">
            This is footer.. Page <pdf:pagenumber/> 
              </div>
        </div>
 
    </body>
    </html>"""
    convert(HTML, 'pisa_example.pdf')

Say thanks to me if this code helps you :) 

 



python feedparser to parse RSS feed

This is very small work but, i wanted to display some feed to my website. The script creates a html file, you can use in your portlet, this is just an example (used rawdog css file).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
import os
import sys
import feedparser
import re
 
MyFeedsConfig = ['http://freshmeat.net/index.atom']
 
feeds = []  
for feed in MyFeedsConfig:  
    feeds.append(feedparser.parse(feed))  
entries = []
fd =  open('output.html','w')
html = """
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
   "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <meta name="robots" content="noindex,nofollow,noarchive">
<meta http-equiv="Refresh" content="10800">
    <link rel="stylesheet" href="style.css" type="text/css">
    <title>FeedParser-V</title>
</head>
<body id="rawdog">
<div id="header">
<h1>FeedParser</h1>
</div>
<div id="items">
 
</div>
<h2 id="feedstatsheader">Feeds</h2>
<div id="feedstats">
<table id="feeds">
<tr id="feedsheader">
<th>Title</th><th>Description</th><th>URL</th><th>Modification Time</th>
</tr>"""
 
for feed in feeds:  
    for item in feed[ "items" ]:
        i = 100
        sumar = item.summary[i]
        while sumar != ' ':
            i = i + 1
            sumar = item.summary[i]
        sumar = item.summary[:i]
        html += """<tr id="feedsrow"><th>%s</th><th>%s...</th><th><a href="%s">go to page</a></th><th>%s</th></tr>""" % (item.title, sumar, item.links[0]['href'], item.updated)
        #print item.content
        #break
html += """</table>
</div>
<div id="footer">
<p id="aboutrawdog">Generated by
Vivek Sharma
</div>
</body>
</html>"""
 
fd.write('%s' % html.encode('ascii', 'ignore'))
    
fd.close()

 



JQuery and python script



Recently i have implemented JQuery in one of the site created through python CGI. I googled a lot about it and got very easy solution to this.
I guess, this can be used in any site where we need to call python script.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$("#AnyID").click(function()
    {
        $.ajax({
            type: "get",
            url: "<your-python-script>.py",
            data: {'param1':'abc'},
            datatype:"script",
            async: false,
            success: function(response) {
            // response is string, convert it to json and apply conditions.
            var json_obj= eval('(' + response + ')');//$.parseJSON("'"+response+"'");
            if (json_obj.type == 'Error'){
              alert(json_obj.msg);
            }
            else {
              alert(json_obj.msg);
            }// else closed
            }, // success closed
            error:function(xhr,err)
            {
                alert("Error connecting to server, please contact system administator.");
            }
        })//ajax closed
}
 


make sure that your python script is callable through url ex. www.example.com/<your-python-script>.py
What i did is, in my <your-python-script>.py script i return dictionary object like {'type':'Success', 'msg':'Your message'}.
If you have CGI script, dont return anything just print the dictionary.

JQuery ajax for python only supports get method for now (i found somewhere, i tried with POST, not working).
here, in above code, you can see "async:false", the purpose of it is very interesting.
While your JQuery run python script and fetch data for you, if you want to block any click event, use "async:true", interesting.. isn't it?



Run Python script method using eval statement

 

Recently i did excellent work around to run python script using eval statement.
   

1
p_obj = eval(<python script name>, globals()),<arguments passed to script> 


    Now, you can run any method inside the script by calling p_obj.run()



python module documentation

I was just sitting idle, so thought that i should write some documentation about my new work. There are lot of python files inside a package and it's a huge project. Each python file consist of lot of class and function definition .

I just wanted to document those class and function definition with  doc header. The problem was,i can not import single file and get "dir(that_file)" or import class of file then do "dir" to get its methods and attributes.

So, i wrote down a simple script which read a single file and result all methods, classes and class methods. This script might be helpfule if you have very big python module and you just wanted to extract out class, method and class methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
import os
import sys
import glob
 
def main(filename):
    """main function """
    new_file = filename.split('.')[0]+'.txt'
    nfl = open('abc.txt','a')
    #import pdb
    #pdb.set_trace()
    class_F = 0
    def_F = 0
    q_doc_flag = 0
    global doc
    doc = []
    fl = open(filename, 'r')
    nfl.write(filename+'\n')
    for line in fl:
        line = line.strip().replace(':','').replace('\n','')
        if not line:
            continue
        if line.lower().startswith('class'):
            doc = []
            doc.append(line+'\n')
            nfl.write(line)
            nfl.write('\n')
            class_F = 1
            def_F = 0 
            continue
        if class_F and line.lower().startswith('def'):
            doc = []
            doc.append('\t'+line+'\n')
            nfl.write('\t')
            nfl.write(line)
            nfl.write('\n')
            def_F = 1
            continue
        if not class_F and line.lower().startswith('def'):
            doc = []
            doc.append(line+'\n')
            nfl.write(line)
            nfl.write('\n')
            def_F = 1
            continue
        if (class_F or def_F) and line == '"""' :
            line = "\t" + line +'\n'
            if line in doc:
                q_doc_flag = 0
                if class_F and def_F:
                    class_F = 0
            else:   
                q_doc_flag = 1
            doc.append(line) 
            nfl.write("\t")
            nfl.write(line)
            nfl.write('\n')
            continue
        elif class_F and line.startswith('"') and line.endswith('"') and line != '"""':
            doc.append("\t" + line +'\n')
            nfl.write("\t")
            nfl.write(line)
            nfl.write('\n')
        elif def_F and line.startswith('"') and line.endswith('"') and line != '"""':
            doc.append("\t\t" + line +'\n')
            nfl.write("\t\t")
            nfl.write(line)
            nfl.write('\n')
        if q_doc_flag == 1:
            doc.append("\t" + line +'\n')
            nfl.write("\t")
            nfl.write(line)
            nfl.write('\n')
 
 
if __name__ == '__main__':
    args = sys.argv
    print args
    if len(args) != 2:
        sys.exit("python test.py <folder path where your script resides>")
    for fn in glob.glob(os.path.join(args[1], "*.py")):
        main(fn)



Java Struts and DBCP object created........never closed.

Recently, i was working on Java Struts. I faced an issue and i suppose it's worth to write down here so that it work as quick help for others.
The problem i faced was "DBCP object created 2012-29-16 09:29:08 by the following code was never closed:"

To understand this problem you must have good knowledge of programming. In my

case, i neither worked on Struts framework nor in java but, have good

understanding of other programming language like Python. I googled a lot about

it, and found a solution somewhere that, it happened only due to unclosed or

abandoned connection pooling.

As the project i was working is very big and its very time taking to readout

each line of code and find out the blunder where the database connection is

opened but not been closed, but to fix this issue i have to do it.

To solve this issue, i did following thing:
1) edit struts-config.xml and added
        <set-property property="removeAbandoned" value="true" />
        <set-property property="logAbandoned" value="true" />
        <set-property property="removeAbandonedTimeout" value="60" />
2) go through your database connection class and functions which access

database, and
        remove all unused ResultSet, PreparedStatement and CallableStatement.
        Optimize your code, if you have opend DB connection inside for loop.

That's all !!!! enjoy.



python quiz program

A very simple quiz written in python with very few lines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
 from random import randint 
 
solved = 0
total_num_q = 0
 
def play(num1, num2, type, solved):
    """ The main play function"""
    def sp_type():
        type = raw_input("Specify the question type(multiplication:M, addition:A, subtraction:S, division:D):")
        if type not in ['M','A','S','D']:
            print "Please input only char M,S,A and D"
        return type
    type = ""
    while type not in ['M','A','S','D']:
        type = sp_type()
    if type == "M":
        ans = input("What's %d times %d? " % (num1, num2))
        result = num1 * num2
    if type == "A":
        ans = input("What's %d plus %d? " % (num1, num2))
        result = num1 + num2
    if type == "S":
        ans = input("What's %d minus %d? " % (num1, num2))
        result = num1 - num2
    if type == "D":
        ans = input("What's %d divided by %d? " % (num1, num2))
        result = num1/num2
 
    if ans == result:
        print "That's right -- well done.\n"
        solved = solved + 1
    else:
        print "No, I'm afraid the answer is %d.\n" % result
    return solved
 
def start_puzzle(solved, total_num_q):
    leval = ''
    play_leval = ['easy', 'intermediate', 'hard']
    while leval not in play_leval:
        leval = raw_input("Which leval you wanted to be in(easy, intermediate, hard): ")
        if leval not in play_leval:
            print "Please enter correct leval name.."
    num_q = input("Please give us the number of question you want to attempt: ")
    try:
        if int(num_q):
            pass
    except:
        print "Please enter only integet value"
    total_num_q = total_num_q + num_q 
    for number_q in range(num_q):
        if leval == 'easy':
            num1 = randint(1, 10)
            num2 = randint(1, 10)
            solved = play(num1, num2, type, solved)
        elif leval == 'intermediate':
            num1 = randint(1, 20)
            num2 = randint(1, 20)
            solved = play(num1, num2, type, solved)
        elif leval == 'hard':
            num1 = randint(1, 30)
            num2 = randint(1, 30)
            solved = play(num1, num2, type, solved)
    return (solved, total_num_q)
 
inp = "yes"
while inp != 'no':
    solved,total_num_q = start_puzzle(solved,total_num_q)
    inp = raw_input("want to start puzzle(yes/no):")    
    
 
print "\nOut of %s questions asked.  You got %d of them right." % (total_num_q,solved)
print "Well done!"



pyjamas - run python script onclick of button

I was designing a web application where i need a simple button to run a bunch of python code resides in Unix system and to show log messages of running scripts to my web application. This can be done in many ways, but i wanted to stick on python and its supported modules.
So, last few days i spent time on learning pyjamas and to configure pyjamas with jsonrpc and mod-python. That really kept me interested as it was totally new to me and lerning pyjamas (i was thinking ot it since a year!!!) is great to me.
I have gone through all sites, tutorials and everything but didn't get any good way to do it on single page and i am sure its really difficult for new programmer to do the same(examples are just to know this can be done but not well explained).

So, i am just writing down things i did to configure pyjamas, jsonrpc and mod python.
The first thing to do is, install jsonrpc, simplejson, mod_python(configured with apache) and pyjamas.

First i design frontend of my application using pyjamas.
1. create a folder VLayouts, then paste following code to VLayouts.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
        import pyjd # this is dummy in pyjs
        from pyjamas.ui.RootPanel import RootPanel
        from pyjamas.ui.SimplePanel import SimplePanel
        from pyjamas.ui.Image import Image
        from pyjamas.ui.Label import Label
        from pyjamas.ui import HasAlignment
        from pyjamas.ui.Button import Button
        from pyjamas.ui.CheckBox import CheckBox
        from pyjamas.ui.VerticalPanel import VerticalPanel
        from pyjamas.ui.HorizontalPanel import HorizontalPanel
        from pyjamas.ui.HTML import HTML
        from pyjamas.ui.DockPanel import DockPanel
        from pyjamas.ui import HasAlignment
        from pyjamas.ui.FlowPanel import FlowPanel
        from pyjamas.ui.HTMLPanel import HTMLPanel
        from pyjamas.ui.ScrollPanel import ScrollPanel
        from pyjamas.ui.DisclosurePanel import DisclosurePanel
        from pyjamas import DOM, Window
        from pyjamas.ui.TextBox import TextBox
        from pyjamas.ui.PasswordTextBox import PasswordTextBox
 
        from FileOpenDlg import FileOpenDlg
        import Utils
        from pyjamas.JSONService import JSONProxy
 
 
        class VLayoutsJSONProxy(JSONProxy):
            def __init__(self):
            ## services/vlayouts.py counts from output folder
            JSONProxy.__init__(self, "services/vlayouts.py", ["callMethod","test"])
            #JSONProxy.__init__(self, "./services/vlayouts.py", ["callMethod","test"])
 
        class VLayoutsService:
            def __init__(self, callback):
            self.callback = callback
            self.proxy = VLayoutsJSONProxy()
 
            def test(self):
            self.proxy.test(self)
 
            def callMethod(self,cmd):
            Window.alert("inside VLayoutsService: callMethod")
            self.callback.showStatus("Calling Method!!!")
            a = self.proxy.callMethod("ls -lth", self)
            self.callback.showStatus(a)
 
            def onRemoteResponse(self, response, request_info):        
            Window.alert("inside VLayoutsService: onRemoteResponse")
            Window.alert(dir(request_info))
            Window.alert(request_info.method)
            Window.alert(request_info.handler)
            Window.alert(response)
            if request_info.method == "callMethod":
                self.callback.showStatus("Called method %s" % request_info.method)
            else:
                self.callback.showStatus(""" REQ METHOD = %s RESP %s """ % (request_info.method,response)) 
 
            def onRemoteError(self, code, errobj, request_info):
            Window.alert("inside VLayoutsService: onRemoteError")
            Window.alert(request_info)
            message = errobj['message']
            Window.alert(message)
            if code != 0:
                self.callback.showStatus("HTTP error %d: %s" % (code, message))
            else:
                json_code = errobj['code']
                self.callback.showStatus("JSONRPC Error %s: %s" % (json_code, message))
 
        class VLayouts(SimplePanel):
            def __init__(self):
            SimplePanel.__init__(self)
            self.VLayoutsService = VLayoutsService(self)
 
            def onModuleLoad(self):
            text="Vivek Sharma"
            self.status = Label()
            contents = HTML(text)
            scroller = ScrollPanel(contents, StyleName="ks-layouts-Scroller")
            panel = DockPanel(BorderWidth=4, Padding=20,Width="100%",Height="100%",
                          HorizontalAlignment=HasAlignment.ALIGN_CENTER,
                          VerticalAlignment=HasAlignment.ALIGN_MIDDLE)
 
            north = Image("./myImage.jpeg",Width="800px", Height="100px",)
            north.addClickListener(getattr(self, "onImageClicked"))
 
            #north  = Label("Image should be placed here")
            west = Button("RUN FUNCTION", self.onButtonClick)
            fileLocation = "logfile.log.txt"
            center = FileOpenDlg(fileLocation = fileLocation)
            ##center = Button("SHOW LOG", self.openLogFile)
            #east   = Label("East")
            #south  = Label("South")
 
            panel.add(north,  DockPanel.NORTH)
            panel.add(west,   DockPanel.WEST)
            panel.add(center, DockPanel.CENTER)
            #panel.add(east,   DockPanel.EAST)
            #panel.add(south,  DockPanel.SOUTH)
            panel.setCellHeight(north, "100px")
            panel.setCellHeight(center, "400px")
            panel.setCellWidth(center, "800px")
            panel.setCellWidth(west, "200px")
 
            panel.setCellVerticalAlignment(west, '100px')#HasAlignment.ALIGN_TOP)
 
            self.add(panel)
            #Window.alert("Hello, AJAX!")
            panel2 = VerticalPanel()
            panel2.add(self.status)
 
            RootPanel().add(panel2)
            RootPanel().add(panel)
 
            def showStatus(self, msg):
            self.status.setText(msg)
 
            def onButtonClick(self,sender):
            Window.alert("onButtonClick function called")
            self.VLayoutsService.callMethod("ls -lth")
 
            def openLogFile(sender):
            fileLocation = "logfile.log.txt"
            dlg = FileOpenDlg(fileLocation = fileLocation)
            dlg.show()
 
            def onShow(self):
            pass
 
            def onImageClicked(self):
            Window.alert("Vivek Sharma!!!")
 
            def makeLabel(self, caption):
            html = HTML(caption)
            html.setStyleName("ks-layouts-Label")
            return html
 
 
        if __name__ == '__main__':
            pyjd.setup("./VLayouts.html")
            app = VLayouts()
            app.onModuleLoad()
            pyjd.run()


configuration i did in /opt/vivek/apache/conf/httpd.conf

        <Directory "/opt/vivek/apache/cgi-bin">
            AddHandler default-handler .jpeg .png .css .html .py .js .txt
            PythonHandler jsonrpc
            AllowOverride All
            Options Indexes FollowSymLinks
            Order allow,deny
            Allow from all
        </Directory>

Now, create .htaccess in /opt/vivek/apache/cgi-bin/VLayouts/public/services folder with content
        Options FollowSymLinks
        PythonPath "['/opt/vivek/apache/cgi-bin/VLayouts/output/services']+sys.path"
        AddHandler mod_python .py
        PythonHandler vlayouts
        PythonDebug On

and check the changes in /opt/vivek/apache/cgi-bin/VLayouts/output/services/.htaccess , or directly create .htaccess here only with above content

create vlayouts.py in /opt/vivek/apache/cgi-bin/VLayouts/public/services folder with following content

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
        #! /usr/bin/env python
 
        import logging
        logging.basicConfig(filename="/home/vivek/Desktop/contactjson.log", level=logging.DEBUG)
        logging.debug("OLDLoading contact service")
 
        from test import AB
 
 
        class Service:
            def callMethod(self, msg):
            return "msg"
 
 
        if __name__ == '__main__':
            # this is if JSONService.py is run as a CGI
            from jsonrpc.cgihandler import handleCGIRequest
            handleCGIRequest(Service())
        else:
            # this is if JSONService.py is run from mod_python:
            # rename .htaccess.mod_python to .htaccess to activate,
            # and restart Apache2
            from jsonrpc.apacheServiceHandler import handler



run as /home/vivek/Desktop/TGZS/pyjamas-0.7/bin/pyjsbuild VLayouts.py



Disc space available

 

Recently i was running a script which uses "/tmp" heavily for all file processing and rolling-up. So at each interval of time i need to know space available to my disc. Looks very simple to run "df -h" every time on console, but i was curious to write python script for this and the result is:

1
2
3
4
5
6
7
8
9
10
11
12
    import os
    import sys
    from time import sleep
    cmd = 'df -h'
    print "Press ctrl-C to exit..\n"
    try:
        while 1:
        print os.system(cmd)
        print '\n'
        sleep(100)
    except KeyboardInterrupt:
        print "Good Bye"



Extract comments in your python code

Comments in your code are always good for new person to understand the logic and flow of program. I guess, people likes programming more, if they comment properly, same way as i do. Proper commenting will not allow you to read code line by line. The best way to comment is to comment in such a way, you can extract it easily. I do comments in my python code using '##' which makes me easy to extract them from script. Following code extracts my comment:


1
2
3
4
5
6
7
8
9
10
11
12
13
import os
import sys
## python extract_comment.py filename comment_delimeter
filename = sys.argv[1]
sepr = sys.argv[2]
fl = open(filename,'r')
fl_con = fl.readlines()
for row in fl_con:
    if sepr in row:
        ind = row.find(sepr)
        print row[ind:]
 
fl.close()



copy the code in file extract_comment.py, filename is the python script from where you need to extract comments and comment_delimeter is '##' in my case.



Simple Web Server in python

Recently, I was hanging arround flex codes which calls python script resides on other server through web services. I got confused, Is it a good idea to use web service just to call python script from other server? Why not to use cgi module or mod-python to get the same result as getting through web services?

So i decided to write a simple web server which has some methods to be called as a URL. Got excellent help from
http://fragments.turtlemeat.com/pythonwebserver.php
then, i added some code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
import string,cgi,time
from os import curdir, sep
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
 
class VivekServer(BaseHTTPRequestHandler):
 
    def do_GET(self):
        try:
        if self.path == '/fetch':
                self.send_response(200)
                self.send_header('Content-type',        'text/html')
                self.end_headers()
                res = self.wcount()
                self.wfile.write("Number of count for 'anyword' :")
                self.wfile.write(res[0])
                self.wfile.write(" url is :")
                self.wfile.write(res[1])
                return
        if self.path == '/calculate':            
                self.send_response(200)
                self.send_header('Content-type',        'text/html')
                self.end_headers()
                res = self.calculate()
                for each in res:
                    self.wfile.write(each)
                    self.wfile.write('\n')
                    return
 
                return
                
        except IOError:
            self.send_error(404,'File Not Found: %s' % self.path)
     
    def calculate(self):
         import random
         WORD = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
         data = []
         for i in range(1, 100):
             data.append((random.randrange(0, 1000), random.sample(WORD, len(WORD))[0]))
         return data
 
    def do_POST(self):
        pass
 
    def wcount(self):
         from BeautifulSoup import BeautifulSoup as soup
         import urllib2
 
         htm  = 'http://www.anyurl.com'
         html_text = urllib2.urlopen( htm ) .read()
         sp = soup(html_text)
         idea = sp.findAll( "anyword" )    
         all_a = [ each.get('href') for each in sp.findAll('a') ]
         num = 0
         for each in all_a:
             if each.find("anyword") > 0:
                 num=num+1
         return (idea,num,htm)
 
def main():
    try:
        server = HTTPServer(('', 7999), VivekServer)
        print 'Server Started.....'
        server.serve_forever()
    except KeyboardInterrupt:
        print 'Server Ends.....'
        server.socket.close()
 
if __name__ == '__main__':
    main()



This is just a help to start a simple web server and call your method through url.

To run above code, just do, python abovecode.py

open web browser, type url as
http://localhost:7999/fetch
http://localhost:7999/calculate 



Split list into number of pieces

Today, while working with csv files i got into fantastic situation where i have a list of million values and i make iteration on that. So i thought, it would be easy for me if i split list into number of pieces which dont affect my code, memory and CPU. I can also use generator expression to make make my code run faster, but i was very curious to write code to split list into number of pieces(uses little of generator exp).
And the result is here -


1
2
3
4
5
6
7
8
9
10
11
12
13
14
    import os,sys
 
    def split_seq(seq, num_pieces):
        """ split a list into pieces passed as param """
        start = 0
        for i in xrange(num_pieces):
            stop = start + len(seq[i::num_pieces])
            yield seq[start:stop]
            start = stop
 
    seq = [i for i in range(100)]   ## define your list here
    num_of_pieces = 3
    for seq in split_seq(seq, num_of_pieces):
        print len(seq), '-> ',seq



Dynamicaly open file

Recently i had gone through a situation like to split a 40GB csv file for further processing, into 60 pieces having name/content to be decided dynamically based on a id present in that csv file. That really make me to write some little code to open/write/close file dynamically. I wrote following code to achieve this.

1
2
3
4
5
6
7
8
9
10
11
import os, sys
a=range(10)
for each in a:
    s = "fl_%s = open('%s','a')" % (each,each)
    exec s
    exec "fl_%s.write('%s')" % (each,each)
    com = "fl_%s.close()" % (each)
    exec com
    # can also check if file is closed or open by
    # com = "fl_%s.closed" % (each)
    # bool(com) #return true if file is closed else false



Join integer value

While working with csv module of python i got very interesting thing about join.
I was reading a huge csv file line by line and for some kind of operation i converted that row to list and again that list to string. But, my row consists of some integer values so i always get
TypeError: sequence item 5: expected string, int found

So, i am writing small code to let new guys know about this.
i have a list and i want to join this.

1
2
ls=['a','b',4,'c']
','.join(ls)


ends up with  : TypeError: sequence item 2: expected string, int found

do,

1
','.join(map(str,ls))



Form submit using Twill

Recently, i found excellent use of Twill module of python. I have used twill before, just to check multiple login functionality of one of my plone site. That was to check load on my login script.
But, some days back, i used Twill to fetch multiple user detail from proxy site.
Here, i am just going to explain the little code, which i wrote. It might be helpful for others.

What i am doing here??

            i am opening google.com and search the term "Twill"    

Here, the first thing is, how to use twill module in python code?
download Twill from http://twill.idyll.org/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import twill
import twill.commands
t_com = twill.commands
## get the default browser
t_brw = t_com.get_browser()
## open the url
url = 'http://google.com'
t_brw.go(url)
## get all forms from that URL
all_forms = t_brw.get_all_forms()         ## this returns list of form objects
 
## now, you have to choose only that form, which is having POST method
 
for each_frm in all_forms:
    attr = each_frm.attrs             ## all attributes of form
    if each_frm.method == 'POST':
        ctrl = each_frm.controls    ## return all control objects within that form (all html tags as control inside form)
        for ct in ctrl:
                if ct.type == 'text':     ## i did it as per my use, you can put your condition here
                        ct._value = "twill"
                        t_brw.clicked(each_frm,ct.attrs['name'])            ## clicked takes two parameter, form object and button name to be clicked.
                        t_brw.submit()
 
## you might write the output (submitted page) to any file using content = t_brw.get_html()
## dont forget to reset the browser and putputs.
t_com.reset_browser
t_com.reset_output




Copyright(c) 2017 - PythonBlogs.com
By using this website, you signify your acceptance of Terms and Conditions and Privacy Policy
All rights reserved