After my initial foray into SQLAlchemy I started the process of mapping my current object model onto database tables and came across issues with my custom, hand-rolled relationship classes. While I expect I could have shoehorned these into SQLAlchemy mappers I decided instead that I'd be better served with an object model that was slightly more integrated with the database structure.
Cue Elixir. Elixir is a thin, ActiveRecord style wrapper on top of SQLAlchemy. It provides some useful features (such as simple to use relationship objects, and polymorphic associations) and allows you to drop down into SQLAlchemy for those features that they haven't wrapped, such as AssociationProxy (for doing a relationship through another relationship). Elixir is the missing link in the stack I was building up, giving me a powerful ORM and database layer with a lovely declarative syntax for building up all the objects I needed.
It took me about a week to go from my old model, through SQLAlchemy and into Elixir but in doing so, I have avoided a whole mess of spaghetti code involved in writing my objects out to a database, obtained database independence, and forced myself to refactor out a lot of code that was proving less than suitable for my requirements. On top of that, I should be able to integrate my system into a pylons or django web application much more easily now than I could have a week ago. Now if only I could get a similar system for XML parsing and allow me to concentrate on the important tasks I still have to do :)
Thursday, 14 May 2009
Friday, 8 May 2009
SQLAlchemy
Continuing on with my sort of 'series' on various libraries and languages, I've recently started to use SQLAlchemy as an ORM layer in my current project.
The reason for using an ORM library came from the immortal problem of suddenly realising that I had a complicated object model defined and an equally complicated database schema and needing to get information between the two of them. I had started to hack together a script that would dump the data in using SQL, but I realised after the first few inserts that it would be a long hard slog that would be inflexible and complicated spaggeti code.
I didn't want to have to rewrite my object model significantly though, which meant I had to find an ORM library that would allow me to set up the mapping myself. I also needed it to work in Python, which then led me to SQLAlchemy. This nifty library comes in several libraries and is very flexible in the right ways to make it suitable for my needs (at least so far).
I'm still in the process of setting up the internal data structures. I'm needing to redo some parts of my internal object system but not significantly so, although I'm worried about it becoming a bit of a mess.
I'll comment on a bit more of the technical side of things in a few days when I've gotten more up to speed with the library.
The reason for using an ORM library came from the immortal problem of suddenly realising that I had a complicated object model defined and an equally complicated database schema and needing to get information between the two of them. I had started to hack together a script that would dump the data in using SQL, but I realised after the first few inserts that it would be a long hard slog that would be inflexible and complicated spaggeti code.
I didn't want to have to rewrite my object model significantly though, which meant I had to find an ORM library that would allow me to set up the mapping myself. I also needed it to work in Python, which then led me to SQLAlchemy. This nifty library comes in several libraries and is very flexible in the right ways to make it suitable for my needs (at least so far).
I'm still in the process of setting up the internal data structures. I'm needing to redo some parts of my internal object system but not significantly so, although I'm worried about it becoming a bit of a mess.
I'll comment on a bit more of the technical side of things in a few days when I've gotten more up to speed with the library.
Friday, 24 April 2009
SAX Parsing
After some time playing with minidom in Python, I finally looked at alternatives and discovered SAX. For my usual requirements (pulling data out of XML files into various internal data structures) this is so much of an improvement over minidom that I wish I'd come to it first, rather than deal with the mess that is minidom for reading XML. I'm still learning the ins and outs of SAX, but as a brief summary for those who know it even less than me I'll explain the basics.
SAX is an event-based XML parser. What this means is that it will read in XML and generate events based on what it has read in. Events would be something like the start of an element, the end of an element, etc. Your application then receives these events and can respond to them appropriately. This can entail a small amount of extra book-keeping in your application to know where you are in the XML tree (assuming that's important for your app), but it is so much easier to deal with than the multiple levels of looping through child nodes that is required to pull data out of an entire DOM tree.
My first problem with SAX (at least from what I know of it so far anyway) is that with a complicated XML file you can end up with an extremely large and non-cohesive handler for your file. To get around this I created a SAX content handler that maintains a 'processor stack' that allows you to split down your processing into more flexible, cohesive units. It isn't perfect (it has some ugliness when it comes to transferring data between processors currently, and adding a new processor is something that is hard-coded into each processor) but it makes for a nice system to me as I can split up my processing to any level I like, the processor stack acts as that extra book keeping I mentioned and it does so transparently. It allows for more flexibility in changing the structure and reuse of components.
All in all, I quite like SAX and I'll probably like it even more if I spend some time on my content processing to make it a more generic solution :)
SAX is an event-based XML parser. What this means is that it will read in XML and generate events based on what it has read in. Events would be something like the start of an element, the end of an element, etc. Your application then receives these events and can respond to them appropriately. This can entail a small amount of extra book-keeping in your application to know where you are in the XML tree (assuming that's important for your app), but it is so much easier to deal with than the multiple levels of looping through child nodes that is required to pull data out of an entire DOM tree.
My first problem with SAX (at least from what I know of it so far anyway) is that with a complicated XML file you can end up with an extremely large and non-cohesive handler for your file. To get around this I created a SAX content handler that maintains a 'processor stack' that allows you to split down your processing into more flexible, cohesive units. It isn't perfect (it has some ugliness when it comes to transferring data between processors currently, and adding a new processor is something that is hard-coded into each processor) but it makes for a nice system to me as I can split up my processing to any level I like, the processor stack acts as that extra book keeping I mentioned and it does so transparently. It allows for more flexibility in changing the structure and reuse of components.
All in all, I quite like SAX and I'll probably like it even more if I spend some time on my content processing to make it a more generic solution :)
Friday, 13 March 2009
Adobe Flex
I finally got around to trying this out over the last few days, and I have to admit I was surprised by how much I enjoyed the experience. To start explaining, I'll first detail a bit about what Flex is.
Flex is the free, open source SDK for creating Flash applications (or rather swf applications). It does away with that annoying 'timeline' concept thats used in Flash animation, replacing it with a very nice, very functional XML layout format with the ability to embed ActionScript 3 code into the layout. Add onto this the Flex Builder IDE (based on Eclipse, and unfortunately not free, but it does have a free 30 day trial version) with extremely useful code completion and all the nice organisation features I'm used to with IDE projects and you have an environment that makes it, speaking plainly, just makes it easy and fun to do things.
So, to start with, my initial thoughts upon trying Flex were dubious. I've never really liked Flash and have always had the mental model of it being an 'animation package' that shouldn't be used for mainstream web applications. I guess Adobe realised this, which is why they hid animation features in Flex to make it more a mainstream programming language. For reasons I don't want to get into here, I had to put this aside and actually try to create something useful with the language.
After about an hour of downloading(1) and then finding the 'Flex in a week' tutorials on Adobe(2) and I was set up, ready to start producing these hated flash applications. About 30 minutes later, I had a grasp of how to layout the XML applications (using the MXML library), how to link that into ActionScript code to do... whatever... and about 10 minutes later, even how to request data from a website and push that into my application to populate list controls. It was easy, despite having never done any AS coding before and having the barest grasp of the syntax. The tutorials didn't even cover the syntax. It assumed you were clever enough to pick it up by osmosis (a welcome relief I can tell you... I don't mind learning a new syntax, but being told for the 50th time that 'this is a for loop, you use it to iterate over something N times' is not something fun) and concentrated on the stuff that would be less familiar to a programmer first coming to flex - specifying a layout with XML, the conventions used, how to link to services, the event model of Flex and how to specify custom events, etc.
So, I haven't gotten through the complete 'week' course yet, but I rushed through the first two-and-a-half days fairly quickly before deciding to create something 'useful'. I decided that what I would try is a custom Flash reader for my wife's hamster comic strip, HAMIC. I used what I had picked up to request the RSS feeds (that I had created, and modified slightly to throw out some extra stuff I needed) and used my newly found knowledge to create the reader in about a day. I then modified it again to add the ability to create and update comics using a similar interface to the web interface on the actual site. I ran into a couple of small issues that helped me understand what the framework does better (such as when controls get instantiated, how bindable data works a bit better and so forth) but overcame them (apart from one small issue with single comics in a category... but we won't mention that ;)). I may end up doing a lot of web-related stuff as a job soon (fingers crossed, I really need a job right now) and this was the first step towards that goal.
It's certainly a far cry from Symbian, and that's almost certainly a good thing :) It's definitely made me re-evaluate my views on Flash technology.
(1) Flex builder 3 trial - http://www.adobe.com/cfusion/entitlement/index.cfm?e=flex3email
(2) Flex in a week video training course - http://www.adobe.com/devnet/flex/videotraining/
(3) The reader I created, for anyone interested. Please be nice to it, it's only a new app - http://flextestsite.workmad3.com/HAMICFlex/index.html
Flex is the free, open source SDK for creating Flash applications (or rather swf applications). It does away with that annoying 'timeline' concept thats used in Flash animation, replacing it with a very nice, very functional XML layout format with the ability to embed ActionScript 3 code into the layout. Add onto this the Flex Builder IDE (based on Eclipse, and unfortunately not free, but it does have a free 30 day trial version) with extremely useful code completion and all the nice organisation features I'm used to with IDE projects and you have an environment that makes it, speaking plainly, just makes it easy and fun to do things.
So, to start with, my initial thoughts upon trying Flex were dubious. I've never really liked Flash and have always had the mental model of it being an 'animation package' that shouldn't be used for mainstream web applications. I guess Adobe realised this, which is why they hid animation features in Flex to make it more a mainstream programming language. For reasons I don't want to get into here, I had to put this aside and actually try to create something useful with the language.
After about an hour of downloading(1) and then finding the 'Flex in a week' tutorials on Adobe(2) and I was set up, ready to start producing these hated flash applications. About 30 minutes later, I had a grasp of how to layout the XML applications (using the MXML library), how to link that into ActionScript code to do... whatever... and about 10 minutes later, even how to request data from a website and push that into my application to populate list controls. It was easy, despite having never done any AS coding before and having the barest grasp of the syntax. The tutorials didn't even cover the syntax. It assumed you were clever enough to pick it up by osmosis (a welcome relief I can tell you... I don't mind learning a new syntax, but being told for the 50th time that 'this is a for loop, you use it to iterate over something N times' is not something fun) and concentrated on the stuff that would be less familiar to a programmer first coming to flex - specifying a layout with XML, the conventions used, how to link to services, the event model of Flex and how to specify custom events, etc.
So, I haven't gotten through the complete 'week' course yet, but I rushed through the first two-and-a-half days fairly quickly before deciding to create something 'useful'. I decided that what I would try is a custom Flash reader for my wife's hamster comic strip, HAMIC. I used what I had picked up to request the RSS feeds (that I had created, and modified slightly to throw out some extra stuff I needed) and used my newly found knowledge to create the reader in about a day. I then modified it again to add the ability to create and update comics using a similar interface to the web interface on the actual site. I ran into a couple of small issues that helped me understand what the framework does better (such as when controls get instantiated, how bindable data works a bit better and so forth) but overcame them (apart from one small issue with single comics in a category... but we won't mention that ;)). I may end up doing a lot of web-related stuff as a job soon (fingers crossed, I really need a job right now) and this was the first step towards that goal.
It's certainly a far cry from Symbian, and that's almost certainly a good thing :) It's definitely made me re-evaluate my views on Flash technology.
(1) Flex builder 3 trial - http://www.adobe.com/cfusion/entitlement/index.cfm?e=flex3email
(2) Flex in a week video training course - http://www.adobe.com/devnet/flex/videotraining/
(3) The reader I created, for anyone interested. Please be nice to it, it's only a new app - http://flextestsite.workmad3.com/HAMICFlex/index.html
Saturday, 14 February 2009
Fresh Start
I've just created this blog as a branch from my original over at Shared Illusions.
This is going to be my programming blog. I'm going to still post to Shared Illusions, but I'm going to try and keep this blog as a discussion of various coding topics and continue with rambling and general stream-of-consciousness over on Shared Illusions.
As for the name, it's basically drawn from this picture that my wife drew the other day:
The caption was initially meant to be a play on the fact that it's a code-monkey reflected and that there is a programming technique called reflection (this part obviously only clear to coders with a fairly poor sense of humour, i.e. me). After reading This post from coding horror a third meaning became apparent. A reflective coder is a programmer who thinks about programming all the time, and uses this to constantly reflect upon and improve their skills.
I'm not a reflective coder yet, but I have the first part down as I never seem to stop thinking about coding (well, apart from when I'm thinking about physics, maths or the latest PTerry book :)). My discussions here will help me, and possibly anyone reading, along the path to being reflective coders and better programmers in general.
This is going to be my programming blog. I'm going to still post to Shared Illusions, but I'm going to try and keep this blog as a discussion of various coding topics and continue with rambling and general stream-of-consciousness over on Shared Illusions.
As for the name, it's basically drawn from this picture that my wife drew the other day:
The caption was initially meant to be a play on the fact that it's a code-monkey reflected and that there is a programming technique called reflection (this part obviously only clear to coders with a fairly poor sense of humour, i.e. me). After reading This post from coding horror a third meaning became apparent. A reflective coder is a programmer who thinks about programming all the time, and uses this to constantly reflect upon and improve their skills.
I'm not a reflective coder yet, but I have the first part down as I never seem to stop thinking about coding (well, apart from when I'm thinking about physics, maths or the latest PTerry book :)). My discussions here will help me, and possibly anyone reading, along the path to being reflective coders and better programmers in general.
Subscribe to:
Posts (Atom)
Posts
