I was asked recently how someone could add new libraries, python programmes etc to their copy of Portable GIS. It’s fairly simple, but here’s a quick guide. I’ll use ogr2osm as an example.
I’m pleased to announce the latest release of Portable GIS. This version (v5.2) has only a couple of changes:
- QGIS 2.8 (I’m going to try and do a release to coincide with each long-term release of QGIS)
- Loader has been updated to the latest version
You can download the setup exe and the md5 checksum here.
Older versions are still available but have been archived to avoid confusion.
As always, please let me know of any problems via the Portable GIS google group.
This version also marks the start of an attempt to create a repository for Portable GIS, which will hold the additional and/or changed files and instructions for rolling your own version. This is something that I have wanted to do for some time, but it’s not ready for release yet. Watch this space!
At the end of November (yes, I know it’s a while ago, life got in the way) I was involved in an event called #HackLancaster, part of a wider project in my home town to try and raise interest in the heritage of the old city area beyondthecastle. This was, I think, a great success on several levels, but it also throw up some issues with archaeological and geospatial data that I hadn’t really considered. This was going to be a much longer post, going through everything I think is wrong with trying to put archaeological data on-line as is, but since it’s been sitting in draft form, I think it’s time to just bite the bullet and get something out there…
So without further ado- my job for #HackLancaster was to take a stuffy old access database, some shape files and some geotiffs, and make this available as an api. Here is the toolkit I ended up using:
- Various scripts for converting access databases to postgresql- collected and tinkered with here and here.
- csvkit for dumping csv to postgresql with the bare minimum of effort.
- Dirt Simple PostGIS API for opening up spatial functionality in my postgresql database over http.
- PGRest for opening up standard non-spatial functionality in the database over http (had I been doing this now, I might have tried PostgREST instead)
- Geoserver for serving historic base maps over WMS, and spatial data over WFS/WMS as needed.
- Gitbook for documentation. You can see my documentation here. Live documentation meant it could be updated and added to as we went along.
Insight Number 1: The one thing I’d take away from this part of the project is that all of these pieces of software were easy to set up, use and configure. As a toolkit for making this kind of data available quickly and easily, you can’t go far wrong here.
Insight Number 2: You need to do more than simply make the data available via an api. Complex relational data structures and archaic terminology are a barrier to non-specialists, no matter how technically competent they are. What’s the solution here- collapse the relational structure somehow? It would be nice to have an automated method in case of updates to the source data.
Insight Number 3: Technologies that seemed like a really obvious fit to a GIS person are not so obvious to everyone else- the historic mapping WMS was completely ignored in favour of Google Maps or MapBox and the spatial RESTful endpoint was only used to convert data from British National Grid to Spherical Mercator, under some duress. Functionality that could have been provided by the spatial RESTful end point was generally fudged.
Insight Number 4: It’s important to make committing code part of the hack. While we asked the teams nicely, not one of them made their code available after the event. In retrospect, since we gave out prizes, we should have made committing their code a prerequisite for winning a prize!
Finally, despite my slight gripes above- the teams involved came up with some extremely nice visualisations of the data- if I can persuade them to make them publicly available I’ll definitely put out some links.
On 20th November this year we held the first PostGIS “Show-and-Tell” Day at the HQ of the British Computing Society. This was the first “official” OSGeo:UK event since hosting FOSS4G 2013 in Nottingham last year- it took us that long to recover from the stress!
We deliberately chose to make the event extremely light and informal, with the minimum of organisation, and this seemed to work pretty well. We had a range of speakers doing both longer talks and lightning talks, and then a session at the end where we discussed useful tools and add-ons for working with PostGIS.
The only caveats we placed on speakers were that the talk should (obviously) include PostGIS or PostgreSQL in some way, and that overt sales pitches would be considered uncool. Luckily our speakers obliged! There was also some really good audience interaction.
Highlights for me included Foreign Data Wrappers, for getting at information stored in other databases or behind other apis, but through PostgreSQL, and Raster processing. I was particularly excited by the Twitter FDW, but it turns out that it doesn’t work with the latest version of the twitter api, and it doesn’t look like a particularly well-maintained piece of code. You can see links to all the talks, and notes from the discussion that we had at the end, at our OSGeo:UK GitHub repository for the event: http://osgeouk.github.io/pgday/.
It transpired that there were a number of other events globally, also celebrating #PostGISDay, and perhaps next year it would be good to coordinate these better.
From an OSGeo:UK perspective, we have plans to hold more of these light-touch events, on a range of subjects (ideas gratefully received). Personally I would like to have charged a nominal fee for attending the event, not to make a profit but so that we could make a small donation back to PostGIS but charging money makes the whole process more complicated! We were extremely lucky in this case to get the venue for free, courtesy of the BCS Location Information Group, and to get the food sponsored by Astun Technology.
All in all, a really fun event, both to organise and to attend- thanks to all the speakers and attendees, and here’s to another event quite soon.
I’m pleased to announce not one, but two new releases of Portable GIS! The first, version 4.2, contains QGIS 2.4 and PostGIS 1.5 and will be the last release to include that version of PostGIS.
The second, version 5, contains QGIS 2.4 and PostGIS 2.1 and all future releases will be based on this. Get them here:
There are two important things about these two releases. The first is that the download size is almost twice what it was previously- this is out of my control and is due to the increased size of the QGIS installation. The second is that version 5.0 should be considered a beta release as it’s very much un-tested. If there are any problems let me know via the Portable GIS google group.
You might also be interested to see a short presentation I did about Portable GIS at GeoMob London in September: http://archaeogeek.github.io/geomob-portablegis/#/
Two weeks ago now saw the return of the OSGIS conference in Nottingham, after a year off in 2013 for FOSS4G. I think there had been mixed feelings about this event; those of us heavily involved in the organisation of FOSS4G 2013 had taken a back seat this year, and with FOSS4G 2014 imminent in Portland, it was clearly going to be a smaller scale get together.
I have to say that overall, my impression is that small is good! Small allows you to chat to everyone, see everything you want to see, and generally enjoy, rather than rush around like a mad man or woman. It was nice to see some new faces, and to see a number of papers from local government and business, belying the idea that OSGIS is primarily an academic event. Thanks as always to the chaps at Nottingham for organising.
Astun had a strong showing at the event, with two workshops and two presentations. My colleague Matt Walker did a workshop on OpenLayers3 and Leaflet, and I did one on WPS and PgRouting (a beginners guide). I did a quick introduction to Portable GIS, and another colleague Antony Scott did a comparison to web servers. You can see the workshops at the Astun Technology GitHub pages. As a slight techy aside, Matt and I collectively decided to try GitBook for preparing our workshops, and we’re both very impressed. If you’re interested in a cheeky workflow for pushing a gitbook directly to GitHub gh-pages, see this gist.
Steven Feldman, another Astun employee/adviser also did a talk entitled “There’s no such thing as a free lunch”, a continuation of an emerging theme in open source geospatial at the moment on getting companies to contribute more, or at the very least acknowledge and thank the open source components they use. As always this was very thought provoking (although I worry a little that making people feel responsible for the software they use might potentially back fire)– you can find Steven’s blog post on the talk here.
OSGIS has always had a strong relationship with the OSGeo:UK local chapter, but unfortunately that’s been a bit inactive since FOSS4G- sometimes it’s hard to know what to do next when you’ve fulfilled one of your primary goals! We’re going to have a go at rebooting the local chapter though now- a more detailed post on this will follow soon. The short version is, I’m back co-chairing after a two year absence, and we’ve got some good ideas going forward. Watch this space!
It’s taken slightly longer than I’d like, but I’ve updated Portable GIS to include QGIS 2.2. You can find a copy of this new version on the portable gis page. I’ve included a zip file of the qgis2 folder for those that don’t want to install a full new version. You should be able to simply copy this over the existing apps/qgis2 folder, but you will lose any personalisations, such as new plugins etc that you’ve installed, so you have been warned!
Note that there’s a slight regression with this version- as I’m no longer claiming that QGIS Server and Map Viewer will work. I’ve had all sorts of trouble configuring these to work in the windows environment, let alone portable gis, and I wanted to get this release out without additional delay. When I get chance, I’ll get it finished, and believe me it will deserve a blog post and fanfare all of it’s own.
Also note that this is my first move away from using Dropbox as my file hosting service. Please bear with me if there are any problems with the link!
Recently I had need to evaluate a Proprietary Desktop GIS (PDG for short) to document the procedure for doing a Thing for a client. To avoid any mud-slinging and name calling , I’m naming neither the PDG or the Thing, I’ll just say that the Thing is something that the PDG claims to be able to do. This is not a blog post excoriating PDGs by the way, it’s a reflection on the virtues of simplicity, good documentation, and being honest and open.
So, I download a trial version of the PDG and spend 2 hours installing and licensing it. During this time I have to consult the documentation on exactly what licensing options I wanted for a TRIAL piece of software. I also have to consult the documentation on exactly how to apply the license. No mind, I get the software installed and working and try and do the Thing. I remember from several years ago, last time I tried to do the Thing with the PDG, that it was slightly tricky, but several versions have been and gone, all of which claim to be able to do the Thing. Consequently though, I cut the PDG a bit of slack when it can’t do the Thing, and I try the work-around. Yes, that still works, though I don’t know how you’d guess that from the error messages or the documentation. It’s not ideal to need two methods of doing the Thing but hey ho. I also cut the PDG some slack when it tells me that I can only do the Thing if I adhere to some very unusual naming conventions, which will mean that, should I need to do this for real, I will have a lot of work to do renaming a bunch of stuff.
Let’s take this up a level. I don’t only need to do the Thing, but also the related Slightly More Complicated Thing (SMCT for short) too. I confess that the documentation doesn’t really say out and out that the PDG can do this, but it certainly implies it. Only, it doesn’t seem to be able to without a license for it’s rather more expensive elder brother, the Proprietary Server GIS (PSG for short). However, to explain this to the client, I will need some documentary proof. I can find blog and forum posts admitting it’s true, and for all I know there might be lots of information in the knowledge base for the PDG and PSG but you have to have a customer number to access this and because I am only evaluating the software, I haven’t purchased it yet, so I don’t have one of those.
So, I ask some questions of colleagues, and while waiting for them to get back to me, I try some work-arounds for the SMCT. Needless to say, they don’t work either.
A colleague finally gets back to me. After some incredulity that the PDG really can’t do the SMCT when everything implies that it can, said colleague, in his other role as a re-seller for the PDG rings them up and asks. “Yes, we can do that” says the first person, let me find a Thing-specialist to explain how. “No, we can’t do that” says the Thing-specialist. “Our reasons are very complicated, but here’s some obscure documentation that actually admits that we can’t do it”. We let the client know the good news.
As I said earlier, this is not a post excoriating PDG, it’s a reflection on the virtues of simplicity, good documentation, and being honest and open.
Reflection One: The whole process of installing the PDG and discovering the various methods of doing the Thing was needlessly over-complicated. This may be due to the long history of the PDG, and the enormous feature-set, but it feels like bloat. Complexity and a huge feature-set do not necessarily equate to quality, and similarly simplicity and a smaller feature-set are not a bad thing.
Reflection Two: Why hide documentation behind what’s effectively a pay-wall? Had I actually been in the market for purchasing this software, I would have given up at that point. Documentation should be freely available to everyone.
Reflection Three: We really should not have needed to get a re-seller to ring up, and speak to two different people, just to get a definitive answer on the capabilities of a piece of software. This is wrong on so many levels.
In my opinion, these points have nothing to do with the license applied to the source code of the software, or the name on the box: Don’t fall prey to Zawinski’s Law, do make your documentation comprehensive and easily accessible, and do be honest about your capabilities. I’d pay good money for that.
Here’s a quick and overdue announcement to say that I’m making a new version of Portable GIS available today, including QGIS 2. Consider this one a beta release, since I really want to upgrade PostGIS and GDAL when I get time. Additional upgrades in this version: Astun Technology’s Loader has been upgraded to the latest version, and Psycopg2 is now included.
Before you click on the link, please take time to read the main Portable GIS page, and also do me the favour of reporting any problems that you find at the portable GIS google group, or via Twitter. If you can provide me with a screenshot of an error, and let me know the exact version of Windows that you’re using, then I’ll do my best to fix.
You can pick up the beta version here. This is a dropbox link, so if it’s not available, then I’ve exceeded my bandwidth for the moment, so please try again later.
I hope to get a release out fairly soon with PostGIS 2.1 and GDAL 1.10, but this is not my day job so bear with me!
There’s a lot more discussion about Open Source GIS these days, which is great (even if it took a global financial meltdown for some people to try it). However, one thing that has bothered me for a while is the language that gets used when discussing it. The short version- this is not about arguing that open source is “better” than proprietary, as some twitter debates I’ve participated in recently have assumed! Comparing open source and proprietary is no longer about comparing apples and oranges- it’s about comparing apples with different labels on the side, and the language we use to make these comparisons should reflect that.
Try it, it’s not as dodgy as you think!
This is usually used by people supporting open source software, but to my mind it actually undermines the point people are trying to make- in that it introduces the possibility of “dodginess” (aka risk) into the discussion. Admittedly a few years ago there was some possibility that people’s experience of free software was something that they downloaded from tucows, but that’s no longer the case. If you want to say something is good, then say it’s good rather than that it’s not dodgy!
It’s still very costly to learn how to use open source software
This one comes up fairly often in discussions about the pros and cons of open source. It’s true that learning any type of software takes time (and therefore money), but when you’re talking about the modern incarnations of QGIS for example, I would argue that the outlay is going to be absolutely comparable to any other established desktop GIS package. If what you mean is “My staff, who have been using Package A for years, will need to invest time in learning Package B”, then that is also just as applicable to any software package under the sun- even when B is actually A.2. If anyone can say truthfully that they coped seamlessly with the switch from Windows XP to a later version, or from Office 2003 onwards, then I will call you a big fibber! It’s fair to say that learning to use command-line tools is tricky if you’re used to a GUI, or that databases are difficult for beginners- but the software license has nothing to do with this. Ironically the reason this gets trotted out when discussing open source is that suddenly people have access to a much greater range of tools, and hence are learning packages (such as server-side databases) that they previously wouldn’t have had access to.
But what about the support?
This is often asked by people enquiring about open source software, but it’s too woolly, and easy to counter with corporate SLA-speak. There’s a huge disconnect between the expectations of corporate clients about what they get when they buy subscription to a proprietary software package, and what would happen if there was an actual problem with the software. Have a look at the small print in a EULA if you’re interested. What the actual end-users want, I would imagine, is a responsive support system where if they have a problem, someone fixes it for them. The difference between proprietary and open source software is purely where you go for the answers.
Going back to the apples and oranges analogy- by using these arguments that are just as applicable to any kind of software, you’re not making a valid comparison. I’m also a firm (possibly naive) believer that invalid comparisons are a lose-lose situation for both proprietary and open source software. By muddying the debate, it’s hard to concentrate on the actual, important differences, or indeed make a proper informed decision. Furthermore, if valid comparisons are made, and there do turn out to be differences in (say) support outcomes (rather than options or contracts or other legalese) then that’s something software suppliers of all persuasions can build and improve on.