Nidelven IT - All about Python, Zope & Plone - and Open Source!

Here you'll find issues related to our services. Mostly about Python, Zope and Plone, as well as hosting-related issues.

"Keeping IT real"






Older entries



Atom - Subscribe - Categories

New webshop available / ny nettbutikk tilgjengelig

So, in English first, then in Norwegian. På engelsk først, så på norsk.

OK, so over the course of some years we've been developing a webshop solution for different customer needs, and for well over a year now we've dedicated one developer to further developing the webshop.

There are various options for a new webshop, anything from an out-of-the-box simple webshop setup without much customization, to the most heavily customized webshop imaginable. So there is a wide range of ways the webshop could be setup, from cheap and standard to good-looking and expensive. What kind of setup you want for your organization depends on budget, and how much money you want to put into having an online presence, an online sales channel.

Please do get in touch if this is interesting, info@nidelven-it.no is where we can be reached for questions and comments.

A very old, but working webshop can be seen at http://www.nidelven-it.no/webshop

--

OK, så over noen år nå har vi utviklet en nettbutikk-løsning for forskjellige kundebehov, og i godt over et år nå har vi dedikert en utvikler for å videreutvikle nettbutikken.

Det er forskjellige valg for den nye nettbutikken, alt fra en hyllevare nettbutikk uten mye skreddersøm, til den mest tilpassede nettbutikken som er tenkelig. Det er et vidt spenn med måter nettbutikken kan bli satt opp på, fra billig og standard (funksjonalitet og utseende), til veldig pent og dyrt. Det oppsettet du ønsker for din organisasjon avhenger av budsjett, og hvor mye penger du ønsker å legge i å være på nett, ha en salgskanal på internett.

Vennligst ta kontakt hvis dette er interessant, info@nidelven-it.no er hvor vi kan nås for spørsmål og kommentarer.

En veldig gammel men fungerende nettbutikk kan ses på http://www.nidelven-it.no/webshop


[Permalink] [By morphex] [Webshop (Atom feed)] [Comments] [26 Feb 12:12 GMT+2]

Merry Christmas, and a happy new year!

Thank you for your business this year, we think 2014 will be an interesting and productive year, we have a webshop that has been developed for over a year now that we want to sell some of, and some other things to do.

If there's anything, please mail us at info@nidelven-it.no

[Permalink] [By morphex] [Manager (Atom feed)] [Comments] [23 Dec 18:36 GMT+2]

Quiet and active period

OK, so it's been quiet here for a while. :)

A lot of time has been spent on developing a new webshop concept, my personal focus has been on learning more about sales, and some marketing as well.

The plan now is to do a lot more sales via another company, so if you as a customer have some SaaS solutions, or even other IT-related products we could be interested in hearing from you.

Sites keep running after hosting was sold to another company, so even though we're pretty much out of that business, our customers were and are well taken care of.

Do come in touch if you need some help around anything web-related, we have a lot of experience after all these years and could help you avoid wasting time and money. And even make more money! :)

[Permalink] [By morphex] [Manager (Atom feed)] [Comments] [20 Nov 17:09 GMT+2]

Some security patching to come

There is a security patch coming up for Zope/Plone tomorrow.

We would recommend to all customers that have a Zope/Plone instance to upgrade/migrate to one of the server packages instead, to avoid the risk of server-wide exploits. Get in touch if you want to improve the security of your site(s), for a relatively small cost.

More information will come later today.

[Later..]  There will be downtime related to this fix, so if your site is not responding, don't panic, it should become available as soon as the necessary work has been done to secure the services.

[Permalink] [By morphex] [Hosting (Atom feed)] [Comments] [17 Jun 11:58 GMT+2]

Shutdown of services not paid for

Customers that it hasn't been possible to get in touch with, and that have not paid for their services, will have their services shut down in the time to come.

If your service has been shut down and you want it up and running again, please contact sysadmin@nidelven-it.no

Thank you.

[Permalink] [By morphex] [Hosting (Atom feed)] [Comments] [11 Jun 12:51 GMT+2]

Jep, Jython and CPython compared, all in a nice little script

So, in relation to the work with Jep ( http://www.nidelven-it.no/weblogs/hosting/blog_entry?id=1368... ) - I've been building a script that can build and setup Jep, Jython and CPython in a directory, and then have a compare.sh script that runs the same code on the 3 different systems.

The installation script is here:

https://raw.github.com/morphex/PythonCompare/master/install....

Here's the output from the generated compare.sh file:

morphex@copyleft-laptop:~/projects/self/jython_jepp_installer6$ ./compare.sh
Starting comparison of Jython, Jepp and CPython..
Starting with Jep..
0.39471411705
0.341079950333
0.333596944809
0.34021282196
0.322955131531
0.322613954544
0.322247982025
0.323844909668
0.319895029068
0.324920892715
Ran in milliseconds: 3878.0
Now Jython..
1.25800013542
0.579999923706
0.436999797821
0.361000061035
0.43799996376
0.287999868393
0.289000034332
0.910000085831
0.289000034332
0.290999889374
Ran in milliseconds: 7576
Now CPython..
0.321110010147
0.312225103378
0.312016010284
0.309517145157
0.30745100975
0.313014984131
0.312664985657
0.312491893768
0.311804056168
0.312137126923
Ran in milliseconds: 3165

As you can see, the Jep and CPython scripts are fairly stable in terms of execution speed while Jython varies a bit but gets faster (probably due to the HotSpot technology). If you want to change the test to something else, you can try "./jdk1.7.0_21/bin/java -jar usr/lib/jython-standalone-2.5.3.jar -m compileall -l ." and mytest.py will be recompiled, along with the other .py files.

Now, feel free to use the install.py script as you like, I think it's a good example of how to setup the whole thing in a specific directory.. there was some hair-pulling to get the entire build process setup so that linking and dependencies were setup the right way.

Fun to work with a mix if Python, Java and Bash scripting for a change. I think being able to use Python to script for example prototypes or even production code is a big win in terms of productivity.. weak typing has its place in rapid application development. :)

[Permalink] [By morphex] [Python-only or > 0.5 Python related (Atom feed)] [Comments] [20 May 13:59 GMT+2]

Python in Java

So, these last couple of days I've been playing around with Jep ( http://jepp.sourceforge.net ), the project that embeds Python and makes it available in Java.. it's been a while since I've worked with Java, dependencies and so on, so been pulling hair a bit to get things working.

But, got it working, and one things I've become more aware of lately is how things run, in terms of speed. So I decided to compare Jep, Jython ( http://www.jython.org ) and (C)Python ( http://www.python.org ).

Here's Jep:

>>> from test import pystone
>>> for x in range(10):
... pystone.main()
...
Pystone(1.1) time for 50000 passes = 1
This machine benchmarks at 50000 pystones/second
Pystone(1.1) time for 50000 passes = 1.01
This machine benchmarks at 49505 pystones/second
Pystone(1.1) time for 50000 passes = 1.01
This machine benchmarks at 49505 pystones/second
Pystone(1.1) time for 50000 passes = 1.01
This machine benchmarks at 49505 pystones/second
Pystone(1.1) time for 50000 passes = 1
This machine benchmarks at 50000 pystones/second
Pystone(1.1) time for 50000 passes = 0.99
This machine benchmarks at 50505.1 pystones/second
Pystone(1.1) time for 50000 passes = 1.01
This machine benchmarks at 49505 pystones/second
Pystone(1.1) time for 50000 passes = 1
This machine benchmarks at 50000 pystones/second
Pystone(1.1) time for 50000 passes = 0.99
This machine benchmarks at 50505.1 pystones/second
Pystone(1.1) time for 50000 passes = 1.01
This machine benchmarks at 49505 pystones/second
>>>

It is fairly consistent when it comes to the number of pystones.

Then trying Jython:

>>> for x in range(10):
... pystone.main()
...
Pystone(1.1) time for 50000 passes = 1.816
This machine benchmarks at 27533 pystones/second
Pystone(1.1) time for 50000 passes = 1.128
This machine benchmarks at 44326.2 pystones/second
Pystone(1.1) time for 50000 passes = 0.569
This machine benchmarks at 87873.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.495
This machine benchmarks at 101010 pystones/second
Pystone(1.1) time for 50000 passes = 0.574
This machine benchmarks at 87108 pystones/second
Pystone(1.1) time for 50000 passes = 0.519
This machine benchmarks at 96339.1 pystones/second
Pystone(1.1) time for 50000 passes = 0.497
This machine benchmarks at 100604 pystones/second
Pystone(1.1) time for 50000 passes = 0.501
This machine benchmarks at 99800.4 pystones/second
Pystone(1.1) time for 50000 passes = 0.575
This machine benchmarks at 86956.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.515
This machine benchmarks at 97087.3 pystones/second
>>>

Which shows a great variation, but improvement, in how many pystones can be calculated per second. I assume this is due to the Java HotSpot technology. How well the HotSpot system works with real-life code and data is a different matter.

Finally, we have (C)Python:

>>> for x in range(10):
... pystone.main()
...
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.85
This machine benchmarks at 58823.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.87
This machine benchmarks at 57471.3 pystones/second
Pystone(1.1) time for 50000 passes = 0.87
This machine benchmarks at 57471.3 pystones/second
Pystone(1.1) time for 50000 passes = 0.85
This machine benchmarks at 58823.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
Pystone(1.1) time for 50000 passes = 0.86
This machine benchmarks at 58139.5 pystones/second
>>>

Which shows a fairly consistent number of pystones, around 58000 per second.

Now, the reason I started looking at Jython and then Jep, is that I'm looking at some Java-based content and development frameworks. I'm not sure if there is a big difference in how you can use Jep and Jython, but these are alternatives that can be used when developing in Java systems. In most cases data needs to be sent back to the Java-based application.

Being able to script things fast in Python is a benefit in some situations for rapid prototyping and so on. As is my experience with Python, the speed in Python can be good enough with the right programming techniques and systems. If the applications run too slowly, then switching between Jep and Jython can be a good idea to see which one actually runs the fastest.

[Permalink] [By morphex] [Python-only or > 0.5 Python related (Atom feed)] [Comments] [15 May 18:07 GMT+2]

100000 downloads, a significant contribution to the Python community

So, yesterday the counter for downloaded PyPi packages created by me tipped 100000 downloads, which was cool. It has been going steadily for days and months now, and it was nice to see that that magic number was surpassed.

You can see the packages here:

http://blogologue.com/morten%20w.%20petersen%20-%20pypi%20pa...

Most of those packages are tied to the Zope and Plone systems, but at least one package is Python-only, the email_backport package:

"The email_backport package is a wrap-up of the email > 4.x module
found in Python 2.5 and above."

I created that to be able to use mailing features across a large range of Plone versions, and it has worked fairly well.

The more I work with coding and programming, the lazier I get, should really get better at looking for right places to put code in other Python projects or as separate Python software packages, but the most important and pressing thing is to get something working and working well with the code it is supposed to run in and on..

With that thought, it would be nice if we in the Python community get some documentation and training on how to split things into packages, and maybe have some coordinators that can point people in the right direction based on the requirements that have to be fulfilled. There is a lot of redundant code and having quality over quantity is probably better in terms of quality of the code and that focus is on a smaller codebase, which would result in less bugs and maybe more documentation and so on for the projects as well.

[Permalink] [By morphex] [Python-only or > 0.5 Python related (Atom feed)] [Comments] [19 Apr 14:21 GMT+2]

Over 100000 downloads of my Python packages, Archetypes and the future

So, yesterday was a good day for me, programming-wise. The various PyPi packages I've created by myself have been downloaded over 100000 times.

I'm not sure exactly what to make of those statistics, but I think it's safe to guess that at least thousands of somewhat technical users have been using software I've developed lately.

You can see the statistics here:

http://blogologue.com/morten%20w.%20petersen%20-%20pypi%20pa...

Is this something to beat my chest about? Yes, I think so. Open Source projects rely on contributions from the community to thrive (most of the time anyway), and it is nice to be able to give something back and ensure the future of the project for the sake of jobs and users.

The most popular package is MegamanicEdit, a set of code and templates that enables creation of content type in an easier and more refactored way. Dexterity is the new thing in Plone and there's a lot of push to get it adopted, but I think there are also a lot of users of Plone out there that have invested in Archetypes-based solutions.. because of that I'm contemplating forking Archetypes or getting some sort of collaboration in the community to make sure that Archetypes-based code can still be used for a time to come, to basically say that we take care of users of Plone and think about compatability even for older software.

Another think is the license for Plone and its packages, but that discussion has been covered a bit on the mailing lists already, the question is how much work is required to make for example Archetypes more BSD-like so that one can sell off-the-shelf software for Plone as well.

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [19 Apr 14:12 GMT+2]

For the next ones, TinyMCE issues after installing another product

So, I've training a new guy to create new products from scratch, and in that process we encountered an issue with TinyMCE not working after Products.MenuNavigation 1.0a2 was installed.

The product was using the Extensions/Install.py approach on a recent version of Plone 4. Things looked OK, but I could see the TinyMCE editor didn't display as it should. It rendered the textarea in HTML as it should with all the buttons and so on, but only a simple text area was displayed. To make things more interesting, using kupu instead as an editor fixed all problems.

After some debugging and cleaning up of unecessary code, I added profiles/default/metadata.xml, types.xml, types directory with MenuPage.xml and skins.xml. That worked.

So if you encounter something strange with TinyMCE after installing other products, check to see if the new product is using a proper GenericSetup approach, I guess this is the way forward anyway, unless you have a product that is spanning a wide number of Plone versions.

And roll back the transactions from the install new product point, I don't think reinstalling a product with a GS setup profile will fix TinyMCE once it has been broken.

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [21 Mar 14:24 GMT+2]

Getting up to speed with Active Directory, LDAP and Plone

So, we're going after a project that involves a lot of user management, and in that, Windows Active Directory and LDAP to authenticate against and update, so that when someone updates their info in Plone, the AD is also updated.

I haven't worked with a lot of Windows technology on the server side, it has almost always been Linux and other *nix variants.

So, to show that we are able to get things going with Active Directory, LDAP and Plone I figured I could setup VirtualBox on my Linux laptop, and install a trial version of Windows Server.

I downloaded Windows Server 2012, setup VirtualBox, booted from the ISO but the process stopped because the CPU I'm using is 32 (48) bit, and Windows Server 2012 requires 64 bit.

So OK, I googled again after Windows Server 32 bit and found the Windows Server 2008 trial ISO and downloaded that.

OK, so booting up the VirtualBox system with the ISO, getting to the install screen.. Windows asks for a product key but that shouldn't be necessary because I'm doing a trial..

To speed up the installation process I'm choosing the standard core server install.. which wasn't that smart, as the regular start menu and such were not available, just got a command prompt.. so back to the installer, install full standard version and we're up and running. :)

After some trying and failing I found that I could add all the necessary Active Directory bits through Administration tools -> Server Manager. OK, so got those installed, and setup a local DNS server.

Also installed Plone via the Enfold windows installer, as well as the Java JRE, so I could install Apache Directory Studio (which I opted to install after reading some nice things about it on the net).

OK, so after taking a break from this setup process I fired up VirtualBox again and couldn't get the network going. So, back to the Windows Server installation process, install Java, Plone, Active Directory again. This time, Active Directory was only setup with the LDAP module, so I worked with the simplest setup possible.

I tried some different configurations in buildout.cfg to get the plone.app.ldap package going, first issue was that python-ldap couldn't install properly so after some fiddling around I used the msi installer from PyPi and got python-ldap installed on \plone42\python.

OK, so far so good. But buildout still tried pulling and installing python-ldap.. after googling I found that the python-ldap's python egg.info directory moved to the buildout eggs folder fooled buildout to not download python-ldap. And after running buildout, I removed that folder so that there couldn't be any conflicts between the two info folders.

OK, so far so good.. tried firing up the instance and got the following error message:

ImportError: DLL load failed: The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log for more detail.

Googling I found that it was missing Visual C++ runtime libraries that were missing.. installing runtime libraries for 2005, 2008, 2010 and 2012 (2008 was the right one) that ImportError message disappeared.

So that went fairly well, but starting up with the instance.exe program again raised an ImportError, on win32file. So OK, I looked around and found pywin32 which should contain that module. Downloaded the installer, but didn't get far in the installation process as the Python that comes with the Plone installer doesn't register itself in the windows registry, and it isn't possible to set an installation path manually. So, OK, was about to copy over win32 modules from the new Python 2.6 installation but saw that win32 etc. were already in the site-packages of the Plone installation..

The plot thickens, and again after some Googling it seemed clear that the issue was starting the Plone instance with 'instance.exe fg' didn't work, while starting Plone through the services control panel would.

Well, well. I googled a bit about Enfold plone and finally figured out that there is some LDAP functionality included in the Enfold Plone Server Windows Installer, so I'm going to go for that. Going for server version 4.5 as it looks like the simplest thing to get installed and test how LDAP works with it.

OK, so I tried a bit of this and that, the point of this exercise was to test that it is possible to connect Plone to Active Directory, for retrieval and updating of data..

So, since I'm on a deadline, I decided to go for a simple, verifiable approach to managing AD content. Found pyad, installed Python 2.7, installed pywin32 using the installer, installed easy_install, used easy_install to install pyad. Also setup AD to run Domain Services.

Created the following script:

START

from pyad import pyad
import time

user=pyad.from_cn('Administrator')

for attribute in user.get_allowed_attributes():
    if 'pass' in attribute.lower(): print attribute

print 'old password', user.get_attribute('userPassword')[0][:]
user.update_attribute('userPassword', 'newPassword-' + str(time.time()))

user=pyad.from_cn('Administrator')
print 'new password', user.get_attribute('userPassword')[0][:]

END

And that worked well. It is nice to know that there is a rather uncomplicated way to get access to AD, using Python.

I see that what was Enfold Server will be Open Source, and the plan is to get a proper Plone system integrated with Active Directory in the way most people do, but a simple fallback as described above is nice to have. It would be nice if someone wrote a thorough tutorial or manual on how to integrate Plone with AD, as AD seems to be the choice a lot of organizations use to manage users, access, software licenses and other information.

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [1 comments] [06 Mar 16:02 GMT+2]

Something that slowed me down a bit today - TinyMCE + old skin

So I got an email from a customer today, having problems with editing news articles and other HTML-content on their site. HTML content was only displayed in a text area, even though the personal settings and plone site were setup to use TinyMCE.

They were upgraded from a prior version that wasn't using TinyMCE as the default editor, so I thought something might be wrong there..

After some work, testing and so on, I finally figured out that the issue was the skin settings in portal_skins - the tinymce folder was listed way below the plone_wysiwyg folder, and then the HTML-code for the editor in the text area was fetched from Plone default, and not TinyMCE.

Pressing customize on the tinymce/wysiwyg_editor template (or something similar) and putting it in the active theme custom folder did the trick, from then on TinyMCE worked well.

Just a tip for the weekend, have a good one. :)

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [01 Mar 17:34 GMT+2]

A walk up the pyramid

So, this Sunday I decided I could play a little around with new technology that I haven't tried much. I've worked with Zope and Plone for years, and it is important stay somewhat in touch with other Python web frameworks - to seize opportunities there as well.

I've seen some buzz about the Pyramid project and especially that it is very promising technically, and lightweight (when something says light-weight in software I think OK, somebody knows what they want and are doing).

So, first thing was to install a fresh Python, as the standard Python setup didn't include easy_install. Installed Python, installed ez_setup.py and then did a 'easy_install -u pyramid'. So far so good. :)

OK, so onto testing this thing out, I followed the first example in this page:

http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch...

and modified it so that the server was running on port 1234, because other ports were taken.

Accessing

http://blogologue.com:1234/hello/man

Gave the result

Hello man!

and yes, that's how easy it was to get up and running. :)

For a while now I've been contemplating building a simple URL shortening service (yes late to the party I know), so I decided I could try to create such a service using Pyramid.

I ended up with this code:

from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response
from pyramid import httpexceptions

global urls
urls = []

def redirect(request):
    raise httpexceptions.HTTPFound(urls[int(request.matchdict['index'])])

def index(request):
    try:
        add_url = request.params['b']
        referer = request.referer
        global urls
        urls.append(referer)
        return Response("""Shortened URL: http://blogologue.com:1234/r%s""" %\
                        (len(urls)-1))
    except KeyError:
        return Response("""<html><head><title>Redirect service</title></head>   
    <body><h1>Redirect services</h1>                                            
    <p>Hello there, welcome :)</p>                                              
                                                                                
    <p>To get started using this URL shorting service,                          
    please drag the following link to your bookmark                             
    bar, and click on the link on whatever page you'd                           
    like bookmarked and accessible via this URL shortening                      
    service.</p>                                                                
                                                                                
    <p><a href="javascript:document.location='http://blogologue.com:1234/?b'"   
         >Shorten URL</a></p>                                                   
    </body>                                                                     
    </html>""")

if __name__ == '__main__':
    config = Configurator()
    config.add_route('index', '/')
    config.add_view(index, route_name='index')
    config.add_route('redirect', '/r{index}')
    config.add_view(redirect, route_name='redirect')
    app = config.make_wsgi_app()
    server = make_server('0.0.0.0', 1234, app)
    server.serve_forever()

Which is running on http://blogologue.com:1234 now. Give it a try, in total I think I spent 1 hour setting up Pyramid and getting this URL shortening service running. That is very interesting and promising, think I'll have to find some project to get more familiar with Pyramid.

[Update..] See in the comments section for an updated code example that works harder to get an actual URL.

[Permalink] [By morphex] [Python-only or > 0.5 Python related (Atom feed)] [1 comments] [20 Jan 10:15 GMT+2]

Learning things the hackish way

I've been studying more Spanish the last couple of months, and take it as I guess a part of the job, to increase our website reach and open up opportunities in one of the major world languages.

So, I've been using various places, products and services to learn Spanish. One of the most recent services I've started to use is Livemocha, and signed up for one of their new year offers for cheap access.

Well, all is good. But then I accessed the website and remembered that, ah, they *require* version 11.3 of the Flash Player, not 11.2 which is the highest version that will be available on Linux.

OK, so after some quick thinking, I thought I could edit the flash player binary plug-in using emacs, because the version string is available in the browser about:plugins page. So searching for the last part of the version string (version was 11.2 r202), r202 through emacs and replacing the 2 with 3 where I could see the 11.2 or 11,2 was, did the trick. I was able to fool the flash plugin and Livemocha that I was indeed running 11.3 and videos etc. worked fine. I read somewhere that Adobe were abandoning Flash but can see now that they have grand plans, just for fewer platforms. It's interesting to see how they are basically shafting a lot of the customers that were depending on Flash as a true cross-platform, browser-based development and runtime environment.

http://www.w3schools.com/browsers/browsers_os.asp shows that Linux is at 4-5%, alone for Linux the numbers are significant, including Android they are staggering.

Anyway, enough about that, I think it's safe to assume they won't be changing Flash or adding new features on minor versions (so 11.<whatever> should be safe). Just fixing bugs, improving performance perhaps.

But, this is a Python post, so to the point. I searched for binary diff for Linux on Google but couldn't find anything useful.

Thinking this was a simple, byte-by-byte comparison I decided I could write my own script, as shown here:

"""
#!/usr/bin/env python
import sys

try:
    script, first, second = sys.argv
except ValueError:
    print 'diff.py firstfile secondfile'

first_data, second_data = open(first, 'r').read(), open(second, 'r').read()

first_size = len(first_data)
second_size = len(second_data)

if first_size != second_size:
    print 'Files differ in size', first_size, second_size

for index in range(first_size):
    first_ = first_data[index]
    second_ = second_data[index]
    if first_ != second_:
        print 'differing in position %s, first has %s, second has %s' % (index, ord(first_), ord(second_))

print 'finished'
"""

When running this script, it gives:

morphex@laptop:/usr/lib/flashplugin-installer$ ./diff.py libflashplayer.so libflashplayer.so.bak
differing in position 21225, first has 51, second has 50
differing in position 15618949, first has 51, second has 50
differing in position 15957965, first has 51, second has 50
differing in position 16008569, first has 51, second has 50
differing in position 16008652, first has 51, second has 50
finished

a list of the differences between the two files. I dropped handling files of different size, as I guess it is a point that things in the file need to be in the same place. Replacing one byte one place in the file with two for example seems like a bad idea, the OS or program using the binary file could depend on correct positions throughout the file.

You can see that the script reads up both files into memory, it would probably be better to read byte-by-byte but file reading, blocking etc. were a bit too much to go into for a simple comparison script, so another argument for skipping files of different size.

So there you go, hope you found this interesting and fun. :)

[Permalink] [By morphex] [Python-only or > 0.5 Python related (Atom feed)] [Comments] [10 Jan 19:32 GMT+2]

Company website upgrade & update, now and then

So, I'm pretty much done with the upgrade of the website

http://www.nidelven-it.no

It's been a bit of work to fine-tune the layout, polish the content (although it should be more). I've also had to go in and modify the plone.app.portlets package to include the suggested patch (don't remember where) that skips <ul> tags in navigation to get an HXTML result that is 100% compliant with the W3C validator.

I started with the beyondskins.responsive theme for layout and design, and have made a lot of changes to it to simplify it, and make it pass the W3C validator. It had some weird HTML-attributes such as role.

There was also an HTML attribute in Plone that didn't jive with the W3C validator, and that was the attribute placeholder on the search box. Isn't (X)HTML compliance a bit important thing for SEO and accessibility, if so - why did something like that slip through the Plone development/release process?

I tested the webpage in a speed tester for counting downloads, download speed and so on. There was a *huge* amount of Javascript included, so I figured I could make Javascript files available only for logged inn users in portal_javascripts.. that worked out well and cut down a big chunk of requests and data transfer. I did the same for portal_kss. Although now I see the search option isn't available for users with a small screen size (probably Javascript) but that'll be fixed.

There was one thing in particular I noticed when validating the page on W3C - and that was that the KSS system put an invalid attribute into the template. So I added a product in products called Patches, with the following __init__.py file:

'''
from zope.interface import implements
from zope.component import getMultiAdapter
from zope.viewlet.interfaces import IViewlet

from Products.Five.browser import BrowserView

class KSSBaseUrlViewlet(BrowserView):
    """ Renders a link rel tag with the real url of the published object. """
    implements(IViewlet)

    def __init__(self, context, request, view, manager):
        super(KSSBaseUrlViewlet, self).__init__(context, request)
        self.__parent__ = view
        self.context = context
        self.request = request
        self.view = view
        self.manager = manager
        self.context_state = getMultiAdapter((context, request), name=u'plone_context_state')

    def update(self):
        pass

    def render(self):
        # We need to generate the link in every case, since the new url base detection
        # algorithm of kss relies on it.
        return u'<link rel="alternate" href="%s/" />' % self.context_state.object_url()

from plone.app.kss import headerviewlet

headerviewlet.KSSBaseUrlViewlet = KSSBaseUrlViewlet
'''

That worked out well.

Anyway, I think the website looks pretty good now, available in English, Norwegian (y un poco) español.

I would *love* to get some comments and feedback on what is good and bad, what should be changed, removed or added.

Thanks :)

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [10 Jan 12:09 GMT+2]

Merry Christmas - happy new year, God Jul - godt nytt år, Feliz Navidad - feliz año nuevo!

We want to wish everyone a merry Christmas. A special thanks to our customer and partners for the collaborations this year and up through the years.

Remember, if you're reading this you're better off economically than most people in the world. Many people don't even have access to basic things like clean water and sanitation.

So now is the time to give, for example lend out money to aspiring entrepreneurs around the world on Kiva, http://www.kiva.org

[Permalink] [By morphex] [Manager (Atom feed)] [Comments] [24 Dec 09:19 GMT+2]

Company website upgrade, day 3.5

I decided that today I was going to launch the site, regardless of small bugs that still were present.

The responsive skin suited us well, but I had to make some final adjustments. I decided the big header image could be dropped from screens that were < 768 pixels wide in styles.css and that the footer info could be dropped from the theme index.html (it has a sharp dark blue) and the page would look cleaner that way.

So far so good.

We had been working on the Webshop in parallel with this, refactoring and moving HTML-y stuff out to templates, and those templates were missing the i18n:domain attribute. So, OK, I have to update some templates with a domain setting for translations. Did that, checked the webshop and nothing. After some thinking and fiddling with styles etc. I remembered it could be compiling the translations files - as nothing else was getting translated as well.

Well, things weren't heavily tested, so I decided to just launch the thing and stick a "(Beta)" besides the logo.

After removing the footer, proper credit to Simples Consultoria wasn't given, so I added an about (om in Norwegian) page for the website where they are credited, as well as some actions which are shown in the upper right corner for contact, about the webpage etc.

Voila, site launched. Have a look at http://www.nidelven-it.no using your computer or any other device. Looks pretty good doesn't it? :)


[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [21 Dec 14:05 GMT+2]

Company website upgrade, day 2&3

So, today I continue on upgrading our company website. We have an internally developed product for simple "page" management, you can add pages, within that page add new pages, images or files. That product will probably released sooner rather than later BTW, probably under the Products.CustomPages name.

Working with the pages product raised the following traceback:

2012-12-19 13:05:03 ERROR portlets Error while determining renderer availability of portlet ('context' '/shared/cop/nidelven/plone' u'navigation'): Object pa-norsk was not found
Traceback (most recent call last):
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.portlets-2.1-py2.7.egg/plone/portlets/manager.py", line 117, in _lazyLoadPortlets
    isAvailable = renderer.available
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.app.portlets-2.2.6-py2.7.egg/plone/app/portlets/portlets/navigation.py", line 135, in available
    tree = self.getNavTree()
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.memoize-1.1.1-py2.7.egg/plone/memoize/instance.py", line 51, in memogetter
    val=func(*args, **kwargs)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.app.portlets-2.2.6-py2.7.egg/plone/app/portlets/portlets/navigation.py", line 204, in getNavTree
    return buildFolderTree(context, obj=context, query=queryBuilder(), strategy=strategy)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.app.layout-2.2.7-py2.7.egg/plone/app/layout/navigation/navtree.py", line 186, in buildFolderTree
    results = portal_catalog.searchResults(query)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.LinguaPlone-4.1.2-py2.7.egg/Products/LinguaPlone/patches.py", line 34, in searchResults
    return self.__lp_old_searchResults(REQUEST, **kw)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.CMFPlone-4.2.1.1-py2.7.egg/Products/CMFPlone/CatalogTool.py", line 428, in searchResults
    return ZCatalog.searchResults(self, REQUEST, **kw)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.ZCatalog-2.13.23-py2.7.egg/Products/ZCatalog/ZCatalog.py", line 604, in searchResults
    return self._catalog.searchResults(REQUEST, used, **kw)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.ZCatalog-2.13.23-py2.7.egg/Products/ZCatalog/Catalog.py", line 907, in searchResults
    return self.search(args, sort_index, reverse, sort_limit, _merge)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.ZCatalog-2.13.23-py2.7.egg/Products/ZCatalog/Catalog.py", line 656, in search
    b_size=b_size)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.ZCatalog-2.13.23-py2.7.egg/Products/ZCatalog/Catalog.py", line 676, in sortResults
    index_key_map = sort_index.documentToKeyMap()
  File "/home/nitwww/plone/instance/buildout-cache/eggs/plone.app.folder-1.0.4-py2.7.egg/plone/app/folder/nogopip.py", line 104, in documentToKeyMap
    pos[rid] = container.getObjectPosition(id)
  File "/home/nitwww/plone/instance/buildout-cache/eggs/Products.Archetypes-1.8.3-py2.7.egg/Products/Archetypes/OrderedBaseFolder.py", line 56, in getObjectPosition
    raise NotFound, 'Object %s was not found' % str(id)
NotFound: Object pa-norsk was not found

This error occurred after I changed the language from Neutral to Norwegian - so as I thought, a portal_catalog clear and reindex operation fixed that issue.

On to day 3.. been trying different themes and it is easy to mess up things, but found, finally, that beyondsskins.responsive could be used. Spent a bit of the day tracking down a bug where portlets were rendered downwards in the middle of the page, and tracked that back to malformed HTML that somehow got past the HTML-parsing engine.

Worked a bit on the Products.CustomPages product (not published yet), putting it into an egg directory structure and adding some features.

So, back to theming. The responsive skin is nice, but the columns weren't covering the whole page as they should. Tried setting width for columns in the responsive CSS - that didn't work. Looked through the HTML code, and found that a CSS file from the drupal theme was referenced. Scratched my head a bit about that one on how to remove it and thought "Aha!" - I can just remove it from the entire setup (using buildout).

I have used the responsive skin earlier when I needed a quick fix for my soccer team. One of the changes made there was that I converted the header image from png to jpg and chose a relatively low quality - making the page much faster to load.

So here I was looking on that the design and that the logo is too small. Yesterday I was working with the Drupal theme, but now I needed a somewhat bigger logo. So I did an undo on the logo.png file, downloaded it into the static theme folder on the FS, adjusted the bg.jpg in the responsive package to blend the colors better with the header (using GIMP, background and radient fill - taking the color from the edge of the header image), and then undid the undo on logo.jpg to have that ready for the Drupal theme.

OK. That's enough I think, time to relax a bit after working hard. :)

Stay tuned for the continued upgrade saga. [;]

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [19 Dec 11:30 GMT+2]

Company website upgrade, day 1

So, I've been wanting to upgrade our website ( http://www.nidelven-it.no ) for a good while now. The biggest thing holding us back has been the webshop we've developed interally and sell - but that has gotten a lot of new stuff as well as fixes and clean ups, so it is about ready to go for production.

So I setup a new site from the unified installer package, worked a bit on the webshop and separating it from older, legacy python eggs and code - installed the latest hotfix and got that working.

Went into the ZMI and hit the Plone upgrade button - it went by fast and voila the site was upgraded.

However, we're now going to use LinguaPlone with the webshop and the website so that was installed too and things seem fine.

But, wherever I went in the site I got strange tracebacks like this:

Traceback (innermost last):
Module ZPublisher.Publish, line 126, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 46, in call_object
Module Shared.DC.Scripts.Bindings, line 322, in __call__
Module Products.PloneHotfix20121106.python_scripts, line 63, in _patched_bindAndExec
Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
Module Products.CMFCore.FSPageTemplate, line 237, in _exec
Module Products.CMFCore.FSPageTemplate, line 177, in pt_render
Module Products.PageTemplates.PageTemplate, line 79, in pt_render
Module zope.pagetemplate.pagetemplate, line 113, in pt_render
- Warning: Macro expansion failed
- Warning: <type 'exceptions.AttributeError'>: Macro widgets/rich does not exist for <ATDocument at index_html>
[...]
Module Products.PageTemplates.ZRPythonExpr, line 48, in __call__
- __traceback_info__: context.widget('text', mode='view')
Module PythonExpr, line 1, in <expression>
Module Products.Archetypes.BaseObject, line 287, in widget
Module Products.Archetypes.Renderer, line 26, in render
Module Products.Archetypes.generator.widget, line 147, in __call__
AttributeError: Macro widgets/rich does not exist for <ATDocument at index_html>

as well as

Module ZPublisher.Publish, line 126, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 46, in call_object
Module Products.CMFFormController.FSControllerPageTemplate, line 91, in __call__
Module Products.CMFFormController.BaseControllerPageTemplate, line 31, in _call
Module Shared.DC.Scripts.Bindings, line 322, in __call__
Module Products.PloneHotfix20121106.python_scripts, line 63, in _patched_bindAndExec
Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec
Module Products.CMFCore.FSPageTemplate, line 237, in _exec
Module Products.CMFCore.FSPageTemplate, line 177, in pt_render
Module Products.PageTemplates.PageTemplate, line 79, in pt_render
Module zope.pagetemplate.pagetemplate, line 113, in pt_render
- Warning: Macro expansion failed
- Warning: <type 'exceptions.AttributeError'>: base_edit
Module zope.tal.talinterpreter, line 271, in __call__
Module zope.tal.talinterpreter, line 343, in interpret
Module zope.tal.talinterpreter, line 867, in do_useMacro
Module zope.tales.tales, line 696, in evaluate
[...]
<Products.CMFFormController.ControllerState.ControllerState object at 0xdcd6b6c>},
'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xe2fb34c>,
'request': <HTTPRequest, URLhttp://orkla.nidelven-it.no:8080/shared/cop/nidelven/plone/i...
'root': <Application at >,
'template': <FSControllerPageTemplate at /shared/cop/nidelven/plone/atct_edit used for /shared/cop/nidelven/plone/index_html>,
'traverse_subpath': [],
'user': <PropertiedUser 'morphex'>}
Module zope.tales.expressions, line 217, in __call__
Module Products.PageTemplates.Expressions, line 147, in _eval
Module zope.tales.expressions, line 124, in _eval
Module Products.PageTemplates.Expressions, line 74, in boboAwareZopeTraverse
Module OFS.Traversable, line 317, in restrictedTraverse
Module OFS.Traversable, line 285, in unrestrictedTraverse
- __traceback_info__: ([], 'base_edit')
AttributeError: base_edit

Now, at first it was hard to get what the issue was, but when the AttributeError base_edit popped up, I got an idea of where to start.

I had a look in

plone/portal_skins/manage_propertiesForm

and found that archetypes, archetypes_kss etc. were all in the different skin configurations so I scratched my head a bit. Well, it turns out that somewhere along the way, the portal_skins/archetypes folder went missing.. So I added a Filesystem directory view in the portal_skins folder and selected the archetypes folder and now things are just fine and dandy.

So, proceeding with the upgrade. I was still getting some weird errors, and decided to do a clear and rebuild of portal_catalog. However, when I got to portal_catalog and tried to update it, various AttributeError exceptions on the attribute clear in ZCatalog were raised. I googled a bit about it and saw suggestions that it could be because of the theme (...) but figured I could copy a portal_catalog from a fresh Plone site instead and delete the one that was failing.

That worked, I now had a working, updated ZCatalog portal_catalog..

So, onwards and upwards. Or so I thought. :)

A new error was raised whenever I tried to switch between languages:

  Module Products.CMFPlone.browser.ploneview, line 295, in have_portlets
  Module plone.app.layout.globals.layout, line 73, in have_portlets
  Module plone.portlets.manager, line 48, in visible
  Module plone.portlets.manager, line 67, in portletsToShow
  Module plone.portlets.manager, line 70, in allPortlets
  Module plone.memoize.view, line 47, in memogetter
  Module plone.portlets.manager, line 107, in _lazyLoadPortlets
  Module plone.portlets.retriever, line 62, in getPortlets
  Module plone.app.portlets.portletcontext, line 40, in globalPortletCategories
  Module plone.app.portlets.portletcontext, line 94, in _getContentType
  Module Shared.DC.Scripts.Bindings, line 193, in __getattr__
Unauthorized: You are not allowed to access 'getTypeInfo' in this context

After some headscratching and looking deep into the portletcontext code and looking that the portal_workflow , I finally figured out that the reason for the failure was that the page I wanted in a different language was in a draft state, and not published (publish would give access to the getTypeInfo method via Access Contents Information in LinguaPlone).

OK, so far so good. That's it for today, will blog later. :)

[Permalink] [By morphex] [Python, Zope, Plone - and anything loosely related to that (Atom feed)] [Comments] [18 Dec 14:02 GMT+2]

Downtime on email

There was some downtime on email this morning, looks like everything is in order now.

[Permalink] [By morphex] [Mail system (hosted by CL) (Atom feed)] [Comments] [01 Dec 12:36 GMT+2]