I recently spent some time trying to build PHP on Windows, I succeeded in the end but it took rather longer than I'd have liked. I normally use the Windows PHP binaries available from snaps.php.net, but in this case I was trying to test whether a proposed patch would fix a Windows specific defect - so I really had to be able to build PHP. I particularly wanted to use a "free" development environment and, as a Windows novice (and life-time *nix user) I hit a lot of simple issues which took a while to solve. I decided to document the steps that I went through in building PHP in the hope that they might be useful to anyone else that would like to build PHP on a Windows system.
Step by step instructions for building PHP6 (and PHP5)
These instructions are intended to cover everything that you need to do to be able to build PHP on a Windows XP system. They may work on other Windows systems, I haven't checked. No prior knowledge is assumed, except how to perform basic operations using the windows command line (eg mkdir, copy) and how to use an editor (eg edit) and no software (other than Windows) is assumed to be installed.
The instructions have been tested with PHP6 and PHP5, the only difference being that any references to ICU can be ignored if you are building PHP5.
Before starting to build anything you will need to download a set of tools from Microsoft. This can take some time, for example it took me 45 minutes to install the build tools whilst in the office working with a high speed ethernet connection. Working at home with my own broadband connection it took me 4.5 hours to do the same thing. I don't recommend a dial up connection if you can possibly avoid it. The build tools will use approximately 500MB of disk space.
Visual C++ 2005 Express Edition compiler
- Get the Visual C++ 2005 Express Edition compiler from here
- Check that you have the right operating system and processor to be able to use the compiler.
- Click download and then run, or save to disk and then run if you prefer.
- Read and accept the licence and all default settings.
- The compiler will be installed here: C:\Program Files\Microsoft Visual Studio 8\
- At the end you will be asked to register - this takes about 5 minutes, if you don't do it something will keep nagging you.
Windows Server 2003 SP1 PLatform SDK
- Get the Windows Server 2003 SP1 Platform SDK, here.
- There are three options, I chose PSDK-x86.exe as I'm running a 32 bit Windows system. The other two are for amd64 and ia64 systems.
- Click download and then run, or save to disk and then run if you prefer.
- Read and accept the license, then click through the next screens.
- Select "typical" install.
- The code will be installed in: C:\Program Files\Microsoft Platform SDK\
- This is a long installation - 20 minutes on my office connection, 2 hours from home.
Note: There is a more recent version of the SDK that I didn't find until after I'd used the one linked above. The newer version can be found here. I haven't tried it yet so can't say whether it works with these instructions.
The .NET Framework 2.0 SDK
- Get the .NET Framework 2.0 SDK from here. Careful - it takes a couple of minutes to start up and can put the installation window behind the one you are working in.
- Click run, then read and accept the license.
- On the installation options screen just select the Tools and Debugger - you won't need samples or doc.
- Take default installation folder (will be put under the same directory as the Visual C++).
- This is a long download - two hours on my home connection.
A utility to extract tar files on Windows
There are a number of these - I used this and it worked fine. I just downloaded it and followed the installation instructions
Getting PHP and its pre-reqs.
First decide what directory structure you want for your build environment, I use e:\zoe\BUILDS so everywhere I refer to that directory you will need to replace it with whatever you have decided to call your build directory.
I found that the easiest way to extract .zip files on Windows was to open them with Explorer and select "extract all files".
At the end of this section the sub-directory structure should look like this:
| |-winbld32 |e:\zoe\BUILDS - |-bindlib | |-icu | |-php6.0YYYYMMDDHHmm (or php5.0YYYYMMDDHHmm if you are building PHP5) |
- Get PHP - I used the PHP snaps site and downloaded the tar.gz file. I copied it to my e:\zoe\BUILDS directory and used the tar utility (step 1) to extract the contents into e:\zoe\BUILDS\php6.0-YYYYMMDDHHmm
Customising the build environment
Before we can build anything, a little customisation is required to make Visual Studio Express work nicely with the SDK and to tell the build tools where to find the libraries that PHP needs to build. For reference, flex and bison are part of win32build directory and ICU is in the ICU directory. If you are building PHP5 you won't need ICU of course.
- Add paths to the vsvars32.bat file
In directory C:\Program Files\Microsoft Visual Studio 8\Common7\Tools\ edit vsvars32.bat to include references to the Platform SDK in PATH, INCLUDE and LIB.
The relevant lines in my copy look like this:
@set PATH=C:\Program Files\Microsoft Visual Studio 8\Common7\IDE;C:\Program Files\Microsoft Visual Studio 8\VC\BIN;C:\Program Files\Microsoft Visual Studio 8\Common7\Tools;C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\bin;C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727;C:\Program Files\Microsoft Visual Studio 8\VC\VCPackages;C:\Program Files\Microsoft Platform SDK\Bin;%PATH%@set INCLUDE=C:\Program Files\Microsoft Visual Studio 8\VC\INCLUDE;c:\Program Files\Microsoft Platform SDK\Include;%INCLUDE%@set LIB=C:\Program Files\Microsoft Visual Studio 8\VC\LIB;C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\lib;c:\Program Files\Microsoft Platform SDK\Lib;%LIB%
- Changes to environment variables.
- Start->Control panel->System->Advanced->Environment variables
- Edit (or add) "path" to add e:\zoe\BUILDS\icu\bin;e:\zoe\BUILDS\win32build\bin
- Edit (or add) "include" to add e:\zoe\BUILDS\icu\include;e:\zoe\BUILDS\win32build\include
- Edit (or add) "lib" to add e:\zoe\BUILDS\icu\lib;e:\zoe\BUILDS\win32build\lib
These changes won't take effect in any windows that you currently have open - you will need to close them and re-open.
Finally - building PHP
- Start up the build environment
- Start -> Visual C++ 2005 Express Edition -> Visual Studio Tools -> Visual Studio 2005 Command Prompt.
- Building resolv.lib.
- In the Visual Studio window, cd e:\zoe\BUILDS\bindlib
- vcexpress (to start the GUI)
- File - > Open -> Project/Solution -> bindlib.dsp
- You'll get a warning about needing to convert bindlib.dsp to the current format. At "Convert and Open", click "yes".
- Now click on Project->Properties->Configuration properties->C/C++/General
- At the top, click "configuration manager" then select "release" and Win32" as the active configurations
- Under additional Include Directories, add (after the dot) ;c:\Program Files\Microsoft Platform SDK\Include and then click OK (this tells it where to find Winsock.h)
- Finally - click on Build and select Build bindlib.
Assuming the the build is successful you can close the GUI - you should find a sub-directory in bindlib called Release. Copy the file resolv.lib from there to e:\zoe\BUILDS\win32build\lib, overwriting the one that's already there.
- Building PHP
- From within your VS C++ window, change directory to the place that you extracted the PHP source
- At the command line, type:
- cscript /nologo configure.js "--with-extra-libs=e:\zoe\BUILDS\icu" "--with-extra-includes=e:\zoe\BUILDS\icu\include" "--without-s
implexml" "--enable-prefix=e:\php6" "--disable-zlib" "--disable-odbc" "--disable-cgi" "--enable-cli" "--without-iconv" "--without-
- nmake install
- Run a test to make sure that the build succeeded
- Create a file called test.php with the following contents <?php phpinfo() ?>
- At the command line, type "e:\php6\php.exe test.php"
Elizabeth Smith Tutorial
The PHP manual
Guillaume Rossolini has translated this into French - see here: here
Guillaume has also tried the newer version of the Microsoft Platform SDK and has confirmed that it works too. The only difference is that the installation folder is: C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2.
Merci beaucoup Guillaume!!
| Added by Z. Slattery , last edited by Z. Slattery on Jun 25, 2007 (view change)
"Going to Chicago; sorry but I can't take you"
Next week I'll be speaking about SCA for PHP at the php|tek 2007
conference in Chicago. We just published our 1.2 release
, which supports a whole load of new SCA bindings (rest-rpc, xml-rpc ..) that have been in beta for a while, so I shall be demonstrating some of those.
I'm told that the conference sold out a while back, but if you are going, do say hello. I'll be wearing my phpwomen
T-shirt, thanks to Elizabeth N and Ligaya!
Caroline Maynard[Read More
I recently returned from the PHP Quebec Conference
where I gave a presentation on the Service Component Architecture (SCA)
. The slides can be found here
. We've been making quite a bit of progress in adding protocol support so I was able to demonstrate bindings for json-rpc
, all of which are available now. Because the conference sessions are 75 minutes long, I was also able to slip in a couple of demos of prototypes for Atom
syndication as well as a custom binding for the eBay soap API
This was the second PHP Quebec conference I have attended, and I thoroughly enjoyed it. The folks of the PHP Quebec Association
do an excellent job, filling three days with some great material from the likes of Rasmus Lerdorf
, Chris Shiflett
, Ilia Alshanetsky
, Rob Richards
, John Coggeshall
, Marcus Börger
, and many more. They also lay on some great entertainment in the evenings. Check out the photo
Cal put up on DevZone
and you'll get an idea of what I mean :-) .Graham Charters
Last week I attended the second UK PHP Conference in London. The organisers had clearly listened to the comments after the first conference, and this year's event was even better. It's the only event of its kind in the UK, and deservedly popular, with a large lecture theatre completely packed.
Like last year, the conference was a one-day, one-track event, and good value for money at £50. It kicked off with Cal Evans, the editor of the Zend dev zone, presenting "My First Mash-Up", getting down with the code to walk us through various ways of tracking a UPS parcel on GoogleMaps with a traditional server-side model and an AJAX model.
Next up was Simon Laws talking about SCA for PHP. Obviously I'm biased but I thought this went really well. Despite being a big lecture theatre, there were a lot of good questions raised throughout the talk, particularly from people who hadn't seen annotations used for dependency injection before: "are annotations bad for performance", "does it work asynchonously", and so on.
After lunch was Kevlin Henney on "Objects of Desire", a talk only loosely about PHP, and one of the highlights. Kevlin is an independent consultant and trainer, specialising in programming languages, OO design, agile development process, patterns and software architecture. He is widely published, but I hadn't heard him speak before. He turned out to be a witty and entertaining speaker, with a lot of wisdom to impart, and deserved the ovation he received.
The big name speaker came next, Rasmus Lerdorf on "Fast and Rich Web Applications with PHP 5". I'd seen Rasmus give a similar talk a couple of times, so there was not much new content for me, but watching him walk through the process of improving the throughput of a simple site one hundred-fold was as impressive as ever, and many of the audience said this was their highlight. Rasmus also got some good questions which were answered .. frankly!
The final wild card was Bill Gaver, a professor of design at Goldsmiths College on "Designing for the Curious Home". Bill and his team go way beyond conventional design issues, resarching some crazy technology ideas, for example wiring up a family home to try and detect the "mood" and then print a daily "horoscope". The one I'd like in my home was the "drift table", which has a viewing window that allows you to float over satellite views of the planet, rather like being in a hot air balloon. Bill says "through these examples, I suggest that computation can go beyond supporting problem solving, consumption and entertainment to enhance a far more varied and subtle range of domestic activities and values". I did notice some muttering around me about the relevance of this talk to the conference, but for me it was much more memorable than many a technical talk.
I had previously whinged to Marcus Baker that it was maybe a bit questionable to select the speakers without a public cfp, but I changed my mind after I'd enjoyed the sessions so much. The personal selection of speakers gave us a more stimulating selection that a committee would have come up with. It could be a new model for computing conferences, akin to the annual Meltdown festival at the South Bank where individual curators (John Peel, Nick Cave, Morrissey, David Bowie, this year Jarvis Cocker) preside over a quirky and personal selection of performances.
After the conference lots of attendees decamped to LivingSpace
, where much beer was consumed, and I was happy to meet a small UK contingent of PHP Women
. The conversation alternated wildly between PHP-related subjects and knitting and crochet. My male colleagues are always complaining about me expecting them to take part in two discussions simultaneously, so I revelled in this. I did have to go through a bit of a post-feminist reevaluation to get to grips with the interest of young women in handicrafts, a subject which I had personally cast off many years ago, but once I'd seized on the key role of the Jacquard loom
in the history of computing as a rationale, my surprise dissolved. I wonder if the UK PHP Conference team would accept a paper on "Babbage and Knitting" as the wacky topic for the 2008 conference? We thought there were about fourteen women at the conference, of 235 attendees. Someone captured us
on Flickr. Later at dinner it was nice to meet Steph Fox in person for the first time, too.
Some other people's photos from the conference are here
. It seems Rasmus had a lot of people throw an arm around him and expect him to smile for the camera :-).
A shameless bit of self-promotion here: I am pleased to say that my Service Component Architecture for PHP article was published in International PHP Magazine this week. It is more or less the same structure and same material as the presentation I gave at Frankfurt, since I wrote them one after another. It is nice to have the words written down, though. Sadly, I was lazy about submitting a photo, so I appear in the guise of International Man of Mystery:
The same issue has an article by colleagues of mine, Graeme Johnson and Zoe Slattery, in which they describe the outcome of a fairly radical experiment. Poor fools, they set out to make a PHP engine using technology already to hand (bits of the Java Virtual Machine, a bit from IBM Research) but absolutely not using the existing engine itself. And this is the hard bit: they wanted to be able still to host the PHP extensions, unchanged. The difficult thing, if I have understood the article correctly, is that the extensions often have quite deep dependencies on the internals of the engine: they depend heavily on macros which in turn expect to know where to find various fields in various structures. This surely must hold back the evolution of engine and extensions since no one can move without breaking something. In the Java world the equivalent interface (JNI = Java Native Interface) manages to keep all the details hidden behind a few opaque handles. Quite an interesting experiment, and an unusual perspective on the PHP engine, I thought.
Matthew Peters[Read More]
We've just made our first release of SCA on the PECL repository. The code has been added into what was the SDO project and which has now been imaginatively renamed to SCA_SDO (you won't believe how much thought went into that...). The main features of this release are:
- inclusion of the Service Component Architecture (SCA) component
- new PEAR packaging to install SCA and SDO as peer directories (PEAR/SCA and PEAR/SDO)
- update to Apache Tuscany C++ SDO revision level 478193
- new function in SDO_DAS_Relational to support multiple root data objects
- new function in SDO_DAS_XML to support CDATA sections (not yet complete)
- fixes for bugs #9287, #9289, #9339
Unfortunately, we were not able to include the json-rpc binding which was demonstrated at the recent Zend and International PHP Conferences. We hope to be able to include this support soon. It's nice to see some positive comments from people who attended those conference presentations. Greg Whitescarver said, "I’m very glad to say that SCA makes exposing services (especially web services) ridiculously easy", and a review on Developpez.com found SCA "...somewhat strange but nevertheless excellent!" (at least that's the Bablefish translation ;-) ).
If you'd like more information on SCA/SDO, please visit the project's community page.
Finally, I'd like to thank Pierre for his help in renaming the SDO project to SCA_SDO, and preserving the old SDO links in the process. Thanks Pierre!Graham Charters
I went to the International PHP 2006 conference in Frankfurt this week and gave a presentation on "Service Component Architecture for PHP". I have uploaded my slides to this blog but have had to split them into two parts, since the PDFs are otherwise too large to upload. I split them into the beginning and end and a block of 10 slides from the middle
I went into Frankfurt for a few hours on the Sunday afternoon. I expected it to be full of skyscrapers, but I was very very wrong; it has instead a pleasant city centre ringed with a few thrilling skyscrapers in the distance. I put a few of my photos up on Flickr.I was struck once again by the contrast between English and German cities: in Germany the public spaces are so much better cared for and more pleasant than they are in England.
The team that ran the conference did a splendid job, as did the hotel itself. I had not realised what an ordered little world I ws living in until I got to Frankfurt airport on a busy Wednesday evening and had to tussle with the real world.
Graham also gave a presentation on SCA for PHP at the Zend conference last week. Once again, despite presenting on the same topic, our presentations are quite different. He does architecture really well; I like to get grubby with hello world in the debugger.
I enjoyed every talk I went to at the conference. I especially enjoy hearing people who know their subject forwards and backwards and who probably wrote the code too, even if I may never need to use what they are talking about, so most of all I enjoyed Tim Bray's talk and the talks by Derick Rethans and Sara Golemon.
Matthew PetersRead More]
I got back from the Zend Conference a few days ago and thought it worth a mention (the conference, not my returning!). I got to attend quite a few session. Rob Richards gave an excellent XML and Web service tutorial. The Amazon and Google presentations were also very interesting. I particularly enjoyed Adam Trachtenberg's pitch on eBay and how to build a business using their API. The example he used was, "Dude, Where's My Used Car" (http://www.dudewheresmyusedcar.com/), a site that he wrote for fun, which mashes eBay Motor listings and Google maps. I'd like to give this a try with SCA, to see whether, a) it works! and b) whether it's any easier. Look out for http://www.excusemesirbutcouldyoupleasetellmewheremyusedmotorvehicleis.co.uk (the thoroughly British subsidiary ;-) ).
Of the main sessions, I found Chris Anderson's Long Tail pitch very interesting and I'm looking forward to reading his book. As ever, Robert "r0ml" Lefkowitz gave an entertaining and thought provoking pitch comparing the evolution of language and literacy over the centuries to the evolution of programming languages and programming literacy. Using this analogy, it appears programming is currently somewhere around the 10th century and programmers are the equivalent of scribes; being paid to read and write for someone else. Another nugget which I believe was attributed to Donald E Knuth's Literate Programming, was the idea of writing programs for humans (the fact that it runs on a computer is incidental). This is something I think we should all do to ensure what we create can be maintained, embraced and furthered by others.
All in all a very good conference, very well organized. A big pat on the back goes to the Zend folks...Graham Charters
Yesterday, I presented and demonstrated SCA for PHP at the Zend/PHP Conference and Expo 2006. You can download the slides here. The session was well attended and I got some good questions at the end. A few people came up asking for the samples I demonstrated (showed how one line of code could be modified to switch from providing a soap/http service to a json-rpc service - I think it's cool, but then I would :-D ).
We are in the process of moving the SCA code into PECL. It will appear in the recently renamed SDO project, which is now called SCA_SDO. I'll put the samples in as part of the move into PECL and blog once this is completed.
I'll blog more about the Zend Conference when I get back...Graham Charters
Service Component Architecture v0.1.0 (alpha) released
We've just made the first release of the Service Component Architecture (SCA) for PHP prototype available.SCA allows you to turn PHP classes into Web services by adding a few simple phpDocumentor-style annotations.SCA will automatically generate WSDL for these services when required. SCA also uses the same annotations technique to allow PHP classes to declare dependencies on other Web services or PHP classes.At runtime it will then 'inject' proxies for these dependencies, which can then be used to call the Web services or classes
Documentation, download package and samples can be found here.A very short whitepaper, which goes through the main concepts of SCA is also available. Our goal is to make this properly open source in the next few weeks, so people can report bugs, submit patches and contribute.We have a phpsoa Google Group where people can discuss SCA and its 'sister' project, SDO.
If you get the chance to give SCA a try, and you have comments/feedback, good or bad, we'd love to hear from you. You can contact us via the Google Group, or via the email addresses for the maintainers of the SDO project.
We recently shipped version 1.0.3 of SDO for PHP
. This was the first version to ship with the C++ SDO library from the Apache Tuscany
open source project - in this case the Milestone 1 release, which was their first packaged version. Currently it's bundled with the SDO for PHP libraries, but as it becomes more stable, we should be able to use Tuscany as an external library too.
Apache Tuscany has both C++ and Java implementations of the SCA specification, as well as SDO. There's a growing community developing lots of exciting new stuff, who post news to http://apache-tuscany.blogspot.com/.
Caroline Maynard[Read More]
Some people who attended my SDO for PHP
talk at php|tek in Orlando wanted to be able to try out the demonstration programs. I've recently uploaded them to CVS
. If you'd like to try them out, copy the whole /scenarios directory under your document root, and browse to it. Some of the commentary is bit terse, so do let us know if you can't make them work or find any problems.Incidentally, we've recently published a new article Streamline working with XML in PHP using Service Data Objects
, which uses the same Half-baked Blog
program shown in the demonstrations to illustrate working with SDOs and XML.
People at the talk had some interesting ideas about how they might use SDOs to solve problems - if you did go home and try them out, I'd be pleased to hear how you got on, and any opinions you'd like to share, positive or negative. I had great time at php|tek, especially hearing about so many diverse areas that people are working in - everyone seems to have something special about their requirements or their environment. I also liked watching the alligators in the lake, and taking my morning swim in the open air (not in the lake).
In recent months we've had some requests from developers who would like to contribute to the SDO for PHP project. We recently added a Contributor License Agreement file to the project, so we can now welcome any developer who would like to be involved. There are instructions in the file about where to send it. Our CLA is based on the Apache CLA, and does not ask you assign the rights to your code, or inhibit you from using it for any other purpose. It simply serves to state the the code is yours to give, and that you do not impose any restrictions on its use.
Caroline Maynard[Read More]
What's the problem?
I recently investigated the values of the PHP and Windows working directory for a variety of environments. My reason for doing so was that I could never quite understand how relative pathnames were resolved, and if you want to write code that will run, for example, both under Zend Studio, from the command line, and Apache, it matters. And it is not surprising that I never understood it either - every environment is different.
PHP working directory
The PHP working directory (from the PHP getcwd() function) will determine how PHP will interpret relative pathnames to PHP functions like PHP’s fopen(), file_get_contents() etc, as well as affect how files are found with include() and require().
Windows working directory
The Windows working directory (obtained with the Windows getcwd() call from direct.h) will determine how relative pathnames are resolved by Windows fopen() calls from within extensions.
Here are the results which illustrate that of the five different environments in which one of our PHP scripts might find itself running, not one of them is the same as any other. In conclusion, be warned: the chances of the Windows and the PHP working directories being the same is slim.
|Environment ||PHP cwd (from PHP’s getcwd()) ||Windows cwd (from windows getcwd() in direct.h)|
|Zend Studio running internal debugger ||location of the script ||C:\Program Files\Zend\ZendStudioClient-5.1.0\bin\php5 |
|Zend Studio running server debugger ||C:\Program Files\Apache Group\Apache2\htdocs ||C:\Program Files\Apache Group\Apache2 |
|php-cli ||same as the Windows cwd|
|normal command line working directory |
|php-cgi ||location of the script ||normal command line working directory |
|Apache ||location of the script ||C:\Program Files\Apache Group\Apache2 |
What does it mean for me
The upshot of this is that you had better be careful which world you are in when you try to do things with files. Here’s something you can usually not do, which you might think you could: use the C library fopen() within an C or C++ extension on a pathname that you were given by PHP. Your PHP script may be able to open the file with PHP’s fopen() but that will be relative to PHP’s working directory and not Windows’. As you see from the table above these are rarely the same.
One thing that was throwing me for a long time is that we were passing PHP-working-directory-based relative pathnames into our SDO extension, which was then passing them on to libxml2 which was happily able to open them. It took a while for it to dawn on me (although I had come across it before) that the PHP libxml extension makes a call to libxml2 when PHP starts, and registers its own file handlers, so that any files that libxml2 needs to work with are actually resolved by the libxml extension, against PHP’s working directory and not Windows’. Thus without realising, our extension was working with PHP-based pathnames. Consequently we could pass the filename to libxml2 and it would be fine, but we could not fopen() the file ourselves.
Perhaps everyone else in the world already knew this and I am just catching up :-)
As promised in my previous entry (albeit a little late), here are the SDO slides
I used at the PHP Quebec conference
The conference was very well organised and there were a good number of very interesting talks. They don't appear to be available on the conference site, but Chris Shiflett's blog has a list of quite a few that are available elsewhere.
The SDO presentation was well received, with some very good feedback. The demos went well and were free from "gremlins", with a number of people expressing an interest in getting hold of the code. I'll see about adding them as samples to the SDO package.
Graham Charters[Read More]
Tomorrow I leave for Montreal to attend the PHP Quebec 2006 conference
. It promises to be a really good event, with presentations from many prominent community members, including Rasmus
I myself will be presenting and demonstrating SDO for PHP
, and given I have a 90 minute slot, I thought it might be an opportunity to look into some of the theory. To that end I've been doing a bit of "design pattern archaeology" - hunting for design patterns after the fact. Armed with Martin Fowler's, "Patterns of Enterprise Architecture"
, and with help from other SDO project members, we quickly identified the following four:
- Data Transfer Object: An object that carries data between processes in order to reduce the number of method calls
- Mapper: An object that sets up a communication between two independent objects.
- Unit of Work: Maintains a list of object affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems.
- Optimistic Offline Lock: Prevent conflicts between concurrent business transactions by detecting a conflict and rolling back the transaction.
There are others, but there's only so far you can take this kind of activity!
The demos I have show how to use SDO to work with relational and XML data. They include a simple Contacts management application, a noddy Blog/RSS feed, and WSDL generation.
If you're not able to make it to Montreal and are interested in SDO, the docs can be found here
, and I'll post the presentation on this blog after the conference.Graham Charters
At last month's Zend Conference I was fortunate enough to attend Christian Wenz's tutorial on XML and Web services. This covered the use of technologies such as DOM and SimpleXML for working with XML data. As the title of this blog entry suggests, SDO provide a simple way to construct or extend XML documents.
The example below shows an XML schema used by an application which records information regarding quotations people have made (Thanks go to Christian Wenz for the scenario which is from his book entitled, "PHP Phrasebook", SAMS Publishing.). An XML document following this schema will contain a quotes
element containing a number of quote
elements, each of which consists of a phrase
and an author
element and a year
<?xml version="1.0" encoding="UTF-8"?>
<xs:element maxOccurs="unbounded" ref="quote"/>
<xs:attribute name="year" use="required" type="xs:integer"/>
<xs:element name="phrase" type="xs:string"/>
<xs:element name="author" type="xs:string"/>
The code extract below shows how SDO can be used to load the XML schema and a quotes
document and then add a new quote
entry to that document.
1 $xmldas = SDO_DAS_XML::create('./quotes.xsd');
2 $xdoc = $xmldas->loadFromFile('./quotes.xml');
3 $quotes = $xdoc->getRootDataObject();
4 $quote = $quotes->createDataObject('quote');
5 $quote->phrase = $_POST['quote'];
6 $quote->author = $_POST['author'];
7 $quote->year = $_POST['year'];
8 $xmldas->saveDocumentToFile($xdoc, './quotes.xml');
The eight lines of code above do the following:
1. create and configure a new XML Data Access Service (DAS) with an XML schema.
2. use the XML DAS to load and validate an instance document.
3. retrieve the quotes currently contained in the document.
4-7. create a new quote and populate it with information posted from a form.
8. save the quotes (including the new quote) back to the xml file.
The same task takes 14 lines of code in DOM and in my opinion, DOM code is far less intuitive. I found it more difficult to understand the data structure from the code, because of the generic XML vocabulary of the DOM APIs. However, it is worth noting that the XML Data Access Service requires the existence of an XML schema, whereas DOM does not.
Because SDO knows about the XML schema, it lets you access the elements and attributes directly by name. The same is true of SimpleXML, however SDO can also be used to create new documents from scratch, or in this case, new fragments of XML. SimpleXML is limited in this regard because it only has knowledge of an instance document, rather than the schema. Again, it is worth noting that SimpleXML does not require the existence of an XML schema.
The SDO API does not reveal which names correspond to XML elements and which to XML attributes, but still outputs XML which follows the provided XML schema. Personally, I think this is quite neat, because often the choice to use an element or attribute in schema design is arbitrary and down to personal taste. This article
nicely sums up the feelings I had when I created my first ever XML schema.Graham Charters
I've had a few requests for the SDO
slides used at the Zend/PHP Conference
. A pdf version of the presentation can be found here
The slides give an overview of the main concepts behind SDOs, followed by a few scenarios showing how to use them to work with relational and XML data. As I mentioned in a previous blog
, the presentation could have done with a few more scenarios to address SDO's strengths and the questions which arose, but unfortunately time was short. I therefore intend to use this blog to cover these over time.
If you have any comments or questions on the SDO project
, please let us know.Graham Charters
I presented an introduction to SDO
for PHP at the Zend/PHP Conference and Expo a couple of weeks ago. The session was very well attended and there were some good questions at the end. Overall the presentation went well, but I could have done with more scenarios to address SDO's real strengths and the questions which arose. To continue the discussion and help clarify a few things, I thought I'd blog a little about SDO. So here goes...
One question which came up during the presentation was to do with the Relational Database Access Service and whether it optimizes the SQL queries it creates in order to remove redundancies. The short answer is, "no", but that's simply because it doesn't have to; SDO does the optimization for it.
The first thing to note is that SDOs keep a change summary, however, this summary only holds the information required to re-create the data object's original state, not any intermediate states. This is important when it comes to understanding how the updates are optimized.
Consider an example of a contact database (as described in this article
). The contact database table contains a "shortname" column (e.g. "Fred") and a "fullname" column (e.g. "Frederick Flintstone"). Let's assume we've retrieved a set of contact SDOs from the database into the variable
. We then perform the following four modifications:
// Create and set a new contact.
// The change summary records the fact that the new contact was created.
$new_contact = $result->createDataObject('contact');
$new_contact->shortname = 'Bertie';
// Delete the new contact.
// The change summary entry for the new contact is cleared.
// It's as if 'Bertie' never existed.
// Change the name of the first contact to "Sally Smith".
// Sally was previously called "Sally Barker", so the old
// name is stored in the change summary.
$result->contact->fullname = 'Sally Smith';
// Change the name of the first contact to "Sally Jones".
// The intermediate value of "Sally Smith" is not recorded
// in the change summary.
$result->contact->fullname = 'Sally Jones';
The create and delete cancel each other out and the intermediate value of "Sally Smith" is not recorded, so the resulting change summary would only show that "Sally Barker" had changed her name to "Sally Jones". Consequently, when the Relational Data Access Service is asked to apply the changes back to the database, the only update it would see and perform is the name change.
The resulting SQL UPDATE statement would look like this:
UPDATE contact SET fullname=? WHERE id=? AND fullname=?
with a parameter list of ("Sally Jones", primary key value
, "Sally Barker")
Useful links:SDO for PHP ProjectSDO for PHP documentationRelational Data Access Service documentationGraham Charters
Next week sees the Zend/PHP Conference and Expo
in San Francisco. It promises to be a great experience, with a goodly number of big industry keynote speakers, lots of hands-on tutorials, and three parallel session streams.
I'm presenting an introduction to Service Data Objects (SDO) on the first day which should give people the basic 101-level understanding of SDO and how to use it with XML and databases. I've included a few scenarios to hopefully keep it *real*. I'm really pleased to have been scheduled first on the parallel sessions, so I can then relax and enjoy the rest of the conference :-) .
I'm really looking forward to finally getting to meet members of the PHP community who've helped us in the creation of the SDO project
. The support and guidance we've received throughout has been fantastic.Graham Charters
Finally PHP 5.1 Beta 2 is live. I'm very excited about PHP 5.1 which is another big step for PHP.
Some of the key improvements of PHP 5.1 include:
* PDO (PHP Data Objects) - A new native database abstraction layer providing performance, ease-of-use, and flexibility.
* Significantly improved language performance mainly due to the new Zend Engine II execution architecture.
* The PCRE extension has been updated to PCRE 5.0.
* Many more improvements including lots of new functionality & many bug fixes, especially in regards to SOAP, streams and SPL.
* See the bundled NEWS file for a more complete list of changes.
Everyone is encouraged to start playing with this beta, although it is not yet recommended for mission-critical production use.[Read More
I feel like singing something Doctor Suessish:
- "Oh, the fun you will have
When you code up a style
And your friends' screens explode;
Boy, then try to smile!"
Or perhaps a limerick:
- "A Web-whacker hight Diplodonicus
Promised styles to amaze and astonish us.
But when the pages he drew
Were an unrelieved blue,
His smile became risus sardonicus!"
And that just about blew my creativity diode for the day.
What it's about is fighting with CSS to come up with arrangements and placements of large elements (images, sidebars, et cetera) that work in a) wildly different window sizes, and b) wildly different browsers. I've had to descend to making the CSS files dynamic, so they could adjust parameters according to things like the browser information. And, of course, I'm doing this by making them PHP scripts.
Perfectionist though I may be (although I doubt you'd ever be able to tell from my office at home), this is one of those all-too-numerous cases in which "right" is going to have to wait upon "good enough." I'll twiddle the styles until it looks the way I want it to do, and maybe some year I'll come back and do the styles right.
And, as usual, my perfectionism means that the scripts are too prototypical to show anybody; I'm too ashamed of their hacky kludgery. But perhaps one day..
The week before last I was in Montreal, Quebec, Canada for PHP Quebec Conference 2005. One of the presentations I gave was about using PHP scripts instead of normally static files like
robots.txt, and it gratifyingly raised a couple of eyebrows.
robots.txt a dynamic file can have a definite impact on performance, but it lets you answer queries a little more specifically. For example, three nasty types of robots are:
- Those which scan without even asking for
- Those which request
robots.txt and either ignore it or use its information to crawl through areas which it explicitly forbids; and
- Those which look at the various clients defined in
robots.txt and then come back disguised as one with more access.
I'm not sure if there actually are any Type III robots out there, but if I can think of it I'm sure some perp somewhere already has as well.
robots.txt is a dynamic file, it can handle Type III malbots by only responding with a single client's permissions those of the client to whose request it's responding. Type I malbots might be caught through the use of spider traps, and Type II can be identified by putting spider traps into a forbidden area and noting that the malbot requested
robots.txt. The fact that it asked for the file and then fell into a trap in a forbidden area is a dead giveaway. :-)
The nasty part of the whole process is reliably identifying the perp. In these days of cable and DSL providers handing out DHCP addresses, labelling a particular IPA as being a perp is only valid until the address is assigned to someone else. And identifying by client identification (the
User-Agent request header field) is unreliable because it's easily spoofed and often good software is used to do bad things.
Of course, sometimes the client ID is a dead giveaway, such as
User-Agent: EmailSiphon, or the IPA might be in a fixed range known to be assigned to people of debatable virtue, but for the most part a lot of eyeballing is going to be necessary to settle on rules. The area is fallow for enhancing the response scripts to understand client/IPA combinations, requests per time t, and other heuristics. And I'm even working on some of those. :-)
One of the concepts that are most difficult for new Web developers to fully grasp, is just how dangerous it is to trust user input. Just in the last week, there've been around a dozen or so different reports of vulnerabilities found in Web applications - mostly all of them revolve around unchecked user input. Because of PHPs dominance in the Web application development world, many of the vulnerable applications were ones written in PHP, which hurt PHPs security track record, even though its not the language which is at fault (the same applications, written in any other language would have suffered from the same vulnerabilities).
The challenge of validating user input is not a simple one. The key to meeting this challenge is attention to details combined with knowledge.
At the end of the day - nothing other than the developer herself can ensure that an unsanitized piece of data finds itself as a part of a filename, and sometimes even a database query, which is why paying close attention to what goes where is important.
But that's actually not enough. Few people fully understand just how little of the Web environment can be trusted. Nowadays, most developers know that you cannot rely on GET or POST variables
to have the values you expect (even if they're inside hidden form values) - but how many of them know that you cannot trust any $_SERVER variable that begins with HTTP (e.g., $_SERVER["HTTP_REFERER"])? These can be fully (and easily) spoofed by the remote users, and must not be trusted. Same goes with cookies - they may not be easily visible for or editable by the average user, but as they're saved on the client-side - a 'malicious hacker', or even down to earth script kiddies, can easily set them to their heart's content. And how about $SERVER_NAME ($_SERVER["SERVER_NAME"]), which actually depends on the Host: header sent by the remote user, and can therefore be spoofed under certain circumstances?
Paraphrasing agent Mulder's immortal words, 'Trust Nothing'.
Interesting links:PHP Security ConsortiumMore on trusting (or not trusting) user inputZeev Suraski
At the suggestion of a friend, I added yet another twist to my watermarking script: rotating the watermark a random amount each time.
While cool in theory, it turns out that the practice is a little more difficult. For one thing, PHP's ImageRotate() function creates a new image resource sized large enough to hold the entire rotated image. Even though I'm using a circular watermark, the image it's in is rectangular. So if you rotate a rectangle and put a bounding box around it, you get a lozenge wider and taller than the original one.
For another thing, getting ImageRotate() to do the Right Thing with the transparency is turning out to be a hassle. I'm working with one of the developers of the PHP image library on figuring that one out; it's unclear whether the issue is my boneheadedness or an actual bug in PHP.
When it's finished, or at least far enough along, I'll post more technical details
One of the most popular articles I've ever written has been about Preventing Image 'Theft'
. I wrote it several years ago, but people are still reading it (evidently) and contacting me about it.
I've recently had call to use this sort of thing myself, and what I've got now is rather more advanced than described in the original article. For instance, now I transparently intercept images 'going offsite' and replace them with a correctly-sized blank box containing text about the copyright. And for images I want to be basically previewable but not really usable (if people want a usable version they need to contact me) I watermark 'em.
Watermarking a digital image means adding information to the existing pixels. It can be involve adding invisible information for identification purposes, such as the Digimarc
technique, or it can be to to visibly degrade the quality of the image, perhaps with a message. I've used both, but it's the latter mechanism I needed most recently.
Watermarking for degradation is an interesting challenge. There's nothing you can do short of actually destroying data to keep a really
determined perp from gatting past your defences, but you can make it pretty difficult.
For instance, the degradation watermarking I set up recently uses a watermark with built-in noise, so there isn't a single same-colour region that can be undone. A perp would have to figure out what the watermark pixels are, pixel by pixel, in order to create a mask to remove it. And since I'm using it on dense JPEG images, that's a little difficult.
In addition, the watermark is repeated across the entire image, and not
at regular intervals. Each one is jigged a bit at random, so no two previews of the same image should be identical. (Well, modulo repeats of the random sequence.) This keeps a perp from figuring that the watermark is repeated at fixed intervals, and using that to help remove it. Of course, if it accesses multiple previews of the same image, it can eventually probably
figure out the pixel settings by comparing them. But I suspect that would be a major chore, too.
Both the replacement-with-notice and the watermarking are done in realtime as part of Apache's response to a Web request. The replacement is very low impact indeed, and doesn't cause performance to deteriorate noticeably, but the watermarking involves actual image manipulation, of megapixel images, and so can
slow things down. So you can use the former on almost any server, but the latter really needs a machine with a lot of oomph to keep visitors happy.
You can, of course, get rid of the performance impact by watermarking the images ahead of time, and sending the results normally. That decreases the random factor, though,
and possibly makes the watermark more easily removed.TANSTAAFL
. Security always costs something
, even if it's only (!) convenience.Ken Coar
I use PHP extensively on almost all of my Web pages. (Just about the only ones that don't use it are on servers that don't have it installed. Mine, of course, all
have PHP installed.) In a lot of cases I store the real content in a database, and PHP pulls it out and formats it; sometimes the content is built directly from scanning files and directories.
This has advantages and disadvantages, of course. On the pro side, the content is visible immediately. On the con side, the content is visible immediately. :-) Having every page be interpreted has definite performance implications and not just on the origin server. Unless care is taken, the dynamic nature of the page can result in it rarely or never being cached, which hits you in the system and
the network. If you pay for your bandwidth by the byte, that can be a huge deal.
Like Sam, I use a mix of languages. For Web pages, I use PHP almost exclusively; for standalone apps I use C, PHP, bash
, or Perl, as seems appropriate. In general I use Perl, but if I'm frobbing a database, chances are I'll use PHP unless there's something in CPAN
that argues for Perl.
When I first got involved with blogging, in December 2002, I decided to write my own software from the ground up. In PHP. Of course, I'm a bit-twiddler, and did it that way so I'd understand what this 'blogging' thing was all about and how it worked. (Almost as soon as I brought it online, Sam challenged me
to take the next step. :-) The result can be seen at my blog
I hope to go into this subject in more detail in the future, and I definitely
intend to say some things about how I'm using PHP to automatically guard my Web servers, but this is just an introductory note after all.
Foremost, we would like to thank the developerWorks editors for giving us the opportunity to christen this blog.
Wed like to begin by saying how excited we are with IBMs announcement of embracing PHP. In a way, this brings closure to our romance with IBM which has its roots in the very birth of PHP, as we know it, about eight years ago. Those who read the fine print at the bottom of the PHP 3.0 CREDITS file
, may remember our joint thanks to Michael Rodeh, who taught our Compilation Techniques course in the Technion, Israel Institute of Technology, and also supervised our university project that was later to become PHP. During this period he played a decisive role in the way PHP history unfolded and today happens to be the Director of the IBM Research Labs in Haifa, Israel. As such, some of the first PHP brainstorming sessions happened within the corridors of IBM and therefore, this turn in events seems very natural.
Needless to say, the announcement doesnt only close a loop to a story that began eight years ago, but opens a brand new and exciting landscape. A company the magnitude of IBM putting its know-how and experience behind PHP is something PHP enthusiasts have been awaiting for a long time. We all followed what happened when IBM embraced Linux. Similar to how Linux was a few years ago, PHP today is a great technology that millions of people already use, and that is growing rapidly. However, it has been lacking the necessary endorsement from a serious industry player such as IBM in order to penetrate the mind share of many corporations and the software industry as a whole. We are confident that just like what happened with Linux, IBMs endorsement will lead to a whole new ballgame for this powerful technology.
In order to fully realize the significance of IBMs involvement in PHP, its important to note that it will not sum up in just a marketing stamp of approval. IBMs support will be backed by contributions of technology, and PHP will benefit from IBMs position in the forefront of the software industry. If previously PHP had to adapt itself to standards written with other languages in mind, the day where standards will evolve around PHP is right around the corner.
At this point you may wonder what exactly this blog will contain. Wonder no further. This shared blog will host thoughts and ideas regarding PHP and Web development at large, coming from people with first hand experience in development and deployment of open source technologies, primarily PHP and Apache. In addition to us, you can expect to read thoughts from evangelists including Sam Ruby, Ken Coar and Mark Pilgrim. In addition, this developerWorks section
will host a variety of white papers and articles dealing with a wide range of PHP-related topics, from technical documents and all the way to roadmap discussions. It should be quite interesting around here!
To conclude our maiden post, wed like to extend a warm gratitude to the PHP community, including (but not limited to) the developers, documenters, bug fixers, quality testers, and anybody else who has contributed to the PHP project throughout the years. Each and every one of you has a share in IBMs announcement. PHP would have never been what it is today without you!Andi Gutmans & Zeev Suraski