domain-driven design background there are a series of domain model patterns that describe objects and objects group built with domain-driven design. aggregates describe cohesive object graph with a single point of entry, called root: the internal objects of the aggregate cannot be persistently references from the outside. the domain classes whose instances are inside aggregates are subdivided into entities and value objects: the former have a lifecycle (like a post or a user), while the latter are just values with methods, equivalent to strings and other domain concepts. a prerequisite of these patterns is the immutability of value objects , which can then be shared between aggregates, just like string instances can be in many languages. value objects such as numbers and colors are modified by calling a method on them that return a new instance: every change to their state should produce a new value object. repositories are collection of aggregates: they model operations such as finding an aggregate or persisting a new one. a great departure of modern ddd from the entity/relationship modelling everyone knows is the duplication of data between aggregates to support new scenarios: it's possible some field or object is repeated in different aggregates. when there is an update to an aggregate, it's not necessarily atomically reflected to the other copies of its data. i'll refer to writing calls for generality, to indicate the command side of the command query sepration, which corresponds to everything that causes a change in state in the domain objects (in opposition to the reading side). events as mail messages thus it has become common to copy data between objects in different aggregates : for example, think of a document and invoice object that share the same start/finish date interval. traditionally this duplication is dealt with by extracting a common object, mapped to a common row in the database, with a name invented on the spot. domain events are an alternative that allows for duplicating these data: they reflect changes happened in a single aggregate, and are sent to other aggregates so that they can update themselves. technically speaking, domain events are plain old $yourlanguage objects, containing the modified data but not related to the orm like the main domain objects. domain events are handy for modelling "when" rules that should always be respected no matter who is writing to an aggregate; moreover, their handling can take place in the same transaction or even in a new one. my skeptic view of events was that it can be unclear which events are communicated between objects. after a while, i accepted that unit tests tell us that; moreover, communicating with events is a further level of abstraction which is unnecessary in simple domains but just a giant observer pattern in others. the underlying idea is that no matter who applies a command or modifies a domain object, we already configured the event handling mechanism so that consistency across aggregates is reached according to our policy defined in the event handlers (which may be immediate consistency, or eventual one. or it may result in sending a mail to a human asking him to review the changes: whatever you want.) the only alternative to propagate changes between aggregates would be to have many collaborators passed to the various repositories, but this solution couples the aggregates with each other in many way, while with events you're forced to define one-way messages. the event generator does not make any assumption about who will listen to the event and if it will be listened to at all: events are a point of decoupling like interface are for object collaboration. and it's not that we call static methods by passing a string. we have a clear contract, a domainevents static class, and we publish interesting events (like createdcar or updatedvoyageplan) as plain old domain object which contain all the information about the update, often even composing the relevant domain object. udi dahan discourages the reference to domain objects, and consider events just special value objects ; indeed as our solution matures we are moving towards simpler objects. this choice may force us to consider just what needs to be inserted in the message instead of a full reference (where and if serialization is used to transmit the event, it's simpler to use a value object in fact). moreover, it avoids possible further accidental writing calls to the domain object originating the event. in the application layer events are published by calling a static domain class: as a result event launchers cannot be decoupled from the event (as in udi dahan's approach). we launch events from the repository after an update has been performed, either by choosing an event class directly (in case of an update or creation) or by collecting the events from a queue on the relevant domain object, usually the root of the aggregate. this was a nice idea from a colleague of mine that let us decouple at least entities from the domainevents static class. for now we do not have the requirement to decouple the handling of events from the transaction , so the application layer (which is over the domain layer) open and commits/aborts a transaction, while reconstituting an aggregate, doing some "writing" work (updating it or executing a command) and saving it. the save triggers the event launch, which may trigger work on other aggregates through the configured handler: in case of an error the whole transaction is aborted, ensuring immediate consistency. so we aren't getting the scaling advantages of deferred handling (we're not interested in that for now), but the simplicity of communicating with events while writing code. dynamic language this a php-specific section: however, domain events are an approach typical of java or .net enterprise applications. we use php classes (or interfaces) for routing the events with instanceof; php is a shared nothing environment, so event configuration is done now on a per-action basis to avoid having to create all the objects handling events on each request. however, we want to move the configuration to the application level , with some lazy-loading: for example, configuring lazy event handlers as methods on factory objects that create the real handler and return it along with the name of the method to call. all communication between aggregates happen in a single process and a single address space (for now), so we don't use a bit of the decoupling properties of events. we map value objects into the relational database either as on the parent entity's table (decomposing their fields onto the entity) or as row of their own table. in any case, we have to ensure immutability via encapsulation and only assignign to $this->anyfield into the constructor. our standard pattern is to define setters as new self($this->field1, ..., $nwfieldvalue, ..., $this->fieldn); where n is a small number of fields. we map all domain object with the hibernate-equivalent doctrine 2. we are investigating how to deal with orphaned value objects, which are not reached anymore by any other entity.

Originally Authored by Dejan Noveski I'm not going to dive into how good or extendible Vim is. If you are reading this article, you probably know that. The thing that makes Vim so good, is the scripting environment behind it called VimL. Using this scripting language, you can write any functionality/plugin you need for Vim. Each plugin you use is written in this language. Here's the best part. You only need very little knowledge of VimL to be able to write plugins, if you know Python (or Ruby). What's a vim plugin anyway A Vim plugin is a .vim script that defines functions, mappings, syntax rules, commands that may, or may not, manipulate the windows, buffers, lines. It is a complete piece of code with some specific functionality. Usually, a plugin consists of several functions mappings command definitions and event hooks. When writing vim plugins with Python, often, everything outside the functions is written in VimL. But those are vim commands and they can be learned fast. In fact, VimL can be learned fast, but using python gives so much flexibility. Think about using urllib/httplib/simplejson for accessing some web service that helps editing in Vim. This is why most of the plugins that work with web services are usually done in VimL+Python. Any prerequisites? You must have vim compiled with +python support. You can check that using the command: vim --version | grep +python Vim package in Ubuntu and it's derivatives comes with +python support. To Work - Vimmit.vim What's better than starting with a simple example? This is a plugin that, when called, will retrieve the homepage of Reddit and will display it in the current buffer. Start by opening "vimmit.vim" file (in vim). Since we are writing python code, its good to check if Vim supports Python: if !has('python') echo "Error: Required vim compiled with +python" finish endif This piece is writen in VimL. It's best if we stick to VimL for things like this, mappings and event hooks. This function will check if Vim has python support or it will end the script with an error message. We continue with the main function Reddit(). This is where we use Python and do the main functionality: " Vim comments start with a double quote. " Function definition is VimL. We can mix VimL and Python in " function definition. function! Reddit() " We start the python code like the next line. python << EOF # the vim module contains everything we need to interface with vim from # python. We need urllib2 for the web service consumer. import vim, urllib2 # we need json for parsing the response import json # we define a timeout that we'll use in the API call. We don't want # users to wait much. TIMEOUT = 20 URL = "http://reddit.com/.json" try: # Get the posts and parse the json response response = urllib2.urlopen(URL, None, TIMEOUT).read() json_response = json.loads(response) posts = json_response.get("data", "").get("children", "") # vim.current.buffer is the current buffer. It's list-like object. # each line is an item in the list. We can loop through them delete # them, alter them etc. # Here we delete all lines in the current buffer del vim.current.buffer[:] # Here we append some lines above. Aesthetics. vim.current.buffer[0] = 80*"-" for post in posts: # In the next few lines, we get the post details post_data = post.get("data", {}) up = post_data.get("ups", 0) down = post_data.get("downs", 0) title = post_data.get("title", "NO TITLE").encode("utf-8") score = post_data.get("score", 0) permalink = post_data.get("permalink").encode("utf-8") url = post_data.get("url").encode("utf-8") comments = post_data.get("num_comments") # And here we append line by line to the buffer. # First the upvotes vim.current.buffer.append("↑ %s"%up) # Then the title and the url vim.current.buffer.append(" %s [%s]"%(title, url,)) # Then the downvotes and number of comments vim.current.buffer.append("↓ %s | comments: %s [%s]"%(down, comments, permalink,)) # And last we append some "-" for visual appeal. vim.current.buffer.append(80*"-") except Exception, e: print e EOF " Here the python code is closed. We can continue writing VimL or python again. endfunction Save the file, source it in vim (:source vimmit.vim) and: :call Reddit() Now, the way we call the function is not so elegant. So we define a command: command! -nargs=0 Reddit call Reddit() We define the command :Reddit to call the function. After adding this, open a new bufer and do :Reddit . Home page will be loaded in the buffer. The -nargs argument states how many arguments the command will take. Function Arguments, Eval and Command Q: How does one access functional arguments? function! SomeName(arg1, arg2, arg3) " Get the first argument by name in VimL let firstarg=a:arg1 " Get the second argument by position in Viml let secondarg=a:1 " Get the arguments in python python << EOF import vim first_argument = vim.eval("a:arg1") #or vim.eval("a:0") second_argument = vim.eval("a:arg2") #or vim.eval("a:1") You can define a function with arbitrary number of arguments by putting "..." instead of argument names. You can access these arguments only by position, and you can mix them with named arguments (arg1, arg2, ...) Q: How can I call Vim commands from Python? vim.command("[vim-command-here]") Q: How to define global variables and access them in VimL and Python? Global vars are prefixed with g:. If you want to define one in your script, best thing to do is check if it exists and if doesn't define it and assign some default value to it: if !exists("g:reddit_apicall_timeout") let g:reddit_apicall_timeout=40 endif You can access it from python using the vim module: TIMEOUT = vim.eval("g:reddit_apicall_timeout") If you want to override this setting, you can write: let g:reddit_apicall_timeout=60 in .vimrc . Additional Notes VimL is pretty easy once you try it. Remember that print works and everything you can do with python, you can do in here. Here you can find the documentation for the vim python module. Vimdoc is the possibly the only resource you will need when writing vim plugins. You can also check this IBM developerWorks article . Now, try to extend "vimmit.vim" so the user is able to choose a subreddit (as a first functional argument). Source: http://brainacle.com/how-to-write-vim-plugins-with-python.html

The @XmlAnyElement annotation enables a property to handle arbitrary XML elements.

Erlang is a language and platform designed by Ericsson, whose scope is supporting distributed, real-time and fault-tolerant computation. It has become famous in the programming world probably as the platform on which CouchDB was developed. Erlang has some peculiarities that make it different from the usual imperative, mainstream languages: it is a dynamic, functional language; even more than Clojure in some aspects. As a result, it forces single assignments and immutability: you cannot change existing variables, only create new ones. All state is kept on the stack in the form of function calls and their parameters. Concurrency and message passing is supported at the language level, as actors (called processes in Erlang). Actor model Every process of the Erlang virtual machine is an actor: actors execute independently and communicate only with one-way messages. Actors can create other actors. When an actor finished its computation, it disappears. Like all paradigms, the actor model is defined by what it takes away from the programmer: no shared memory between processes. No remote procedure call: only messages sent via asynchronous invocations. As a result, no return values since messages go in one direction (although of course this can be simulated with another, separate message.) Note that Erlang processes run inside one or more virtual machines on one or more nodes of a network, so they do not map to OS processes. An echo server Without taking too much consideration for Erlang's syntax, let's see it in action. An echo server is a process sending you back the messages you send to it. This first example will serve to use both to learn a bit of Erlang's look&feel, and to introduce the primitives for creating actors. -module(echo). -export([start/0]). loop() -> receive {Sender, Num} -> Sender ! Num, loop() end. start() -> spawn(fun loop/0). This is contained in a echo.erl file, where we define a module named echo. All identifiers starting with lowercase are atoms, while variables start with an uppercase letter. This means Num is a variable, while num would be a constant symbol that cannot be assigned but only passed around (like Ruby symbols or Clojure keywords). We define two functions loop() and start(), but export only start/0 (the start version with 0 arguments); the complete signature of a function always comprehends its arity (the number of arguments), so start() and start(Variable) would be two different functions. In loop/0, we use the receive primitive to receive one new message from the queue for this process. We define a pattern for the message - we only accept tuples containing two variables. After receival, we send back a message to the process identified by the first half of the tuple, containing the number in its second half. We then restart listening by tail recurring on loop() until a new message is delivered. start/0 is instead a primitive that has to be called from the originating process. It would call spawn/1 to create another process which will execute the loop function; it will then return the control to the calling process. Here's how to use this echo server. The shell will be our originating process, while the process for echo will be created: [giorgio@Desmond:~]$ erl Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) We compile the echo.erl source file: 1> c(echo). {ok,echo} We call start() from this process. Another process is spawned, but spawn() returns its process id. Since it is the last statement of start(), also start() returns the same value, which we can save in a variable. The value of the id is shown, as all other return values of local function calls: 2> Pid = echo:start(). <0.39.0> Now we send a message to Pid, a tuple containing two values. A tuple is a simple data structure containing a fixed number of values (it is the generalization of a pair or of a triplet): 3> Pid ! {self(), 42}. {<0.32.0>,42} Now the other process will handle the message and send one back to us. When we want to receive it, we can invoke receive with an identity handler, just displaying what we have got back: 4> receive Value -> Value end. 42 Let's complicate this a bit Let's try to encapsulate some state into our server. We code a new module called var, which will represent a number we can add to or display. -module(var). -export([init/0, add/1, get/0]). init() -> Pid = spawn(fun() -> loop(0) end), register(variable, Pid). % we keep track of the process id loop(N) -> receive {add, X} -> loop(N+X); % we accept more than one type of message: the first has the atom add as the first element for identification {Parent, get} -> % in case of get, before restarting the loop we send a message to Parent with the value Parent ! N, loop(N) end. add(X) -> % these primitives can be called from the parent process, like init() variable ! {add, X}. get() -> variable ! {self(), get}, receive Result -> Result end. The server has the same structure, but it accepts more than one type of message, distinguishing between them via pattern matching. Moreover, it performs some operations like Parent ! N before tail recurring. The add/1 and get/0 primitives are meant to be called from the client process, and encapsulate the messages to send to the server process, along with the blocking for receiving results. To manage multiple created processes, we would have to save pids in a list. Conclusion As you know, different approaches to concurrency such as functional programming and the Actor model are likely to play a role in the near future due to the rise of parallel and distributed computing. To delve into this new world, I'm currently test-driving a distributed tuple space (like JavaSpaces) in Erlang: the mix of a functional language and the actor model is a big change for an OO developer to deal with.

This is an experiment for a real-time tracking of a football match using only web technologies. In this case I’ve chosen to track a football match from Futbol Club Barcelona (fcb). All the work is made by the browser and it’s interactive (there are four checkboxes to activate/deactivate each tracker). It is based on tracking the location of colored objects in the frames using the HSV colour space. As a practical use for the tracker, I’ve added a smart volume feature that activates the sound only when there are goal chances (every time the ball is near the area). So you can browse other pages and change to the video one only when chances are happening. It works based on the quantity of green color in the sides of the video. The path tracker shows ‘possible’ passes between fcb players If you want to watch it in action here is the demo So far it only works in chrome and firefox and because of the low resolution of the video (and my limited skills), the tracking is not perfect, but it could be improved for tracking videos from external sources like Youtube, Veetle, etc. Here is the code: function rgbToHsv(r, g, b){ r = r/255, g = g/255, b = b/255; var max = Math.max(r, g, b), min = Math.min(r, g, b); var h, s, v = max; var d = max - min; s = max == 0 ? 0 : d / max; if(max == min){ h = 0; // achromatic }else{ switch(max){ case r: h = (g - b) / d + (g < b ? 6 : 0); break; case g: h = (b - r) / d + 2; break; case b: h = (r - g) / d + 4; break; } h /= 6; } return [h, s, v]; } //Main object var processor = { isFirefox35: function() { // Let a chance to the navigator to deal with it: return true; var ua = navigator.userAgent; // Gecko ? if (ua.indexOf("Gecko") == -1) return false; // Geck >= 1.9.1 ? return !(ua.indexOf("rv:1.9.1") == -1 && ua.indexOf("rv:1.9.2") == -1); }, // Init doLoad: function() { if (!this.isFirefox35()) { document.getElementById("nofirefoxbeta").style.display = "block"; } // Some init this.displayBackground = true; this.video = document.getElementById("video"); this.mirrorVideo = document.getElementById("mirrorVideo"); this.mirrorVideoCtx = this.mirrorVideo.getContext("2d"); var self = this; // If the videos end, play again this.video.addEventListener("ended", function(){ try { clearTimeout(self.timeout); } catch(e){} self.video.play(); // Work around: https://bugzilla.mozilla.org/show_bug.cgi?id=488287 self.videoIsPlaying(); }, true); // Set the events listeners for the main video (update button) this.video.addEventListener("pause", function() { self.updateButtons(false); }, false); this.pageLoaded = true; this.startPlayer(); }, videoIsPlaying: function() { this.updateButtons(true); this.timerCallback(); }, videoIsReady: function() { this.videoLoaded = true; this.startPlayer(); }, startPlayer: function() { if (!this.videoLoaded || !this.pageLoaded) return; document.getElementById("wait").style.display = "none"; document.getElementById("player").style.display = "block"; this.width = this.video.videoWidth; this.height = this.video.videoHeight; this.mirrorVideo.width = this.width; this.mirrorVideo.height = this.height; }, playVideo:function(){ this.video.play(); this.videoIsPlaying(); }, stopVideo: function(){ this.video.pause(); clearTimeout(this.timeout); }, volumeDown:function(){ if(this.video.volume>0) this.video.volume=Math.round((this.video.volume - 0.1)*10)/10; }, volumeUp:function(){ if(this.video.volume<1) this.video.volume=Math.round((this.video.volume + 0.1)*10)/10; }, // Main loop timerCallback: function() { if (this.video.paused || this.video.ended) { return; } this.computeFrame(); var self = this; this.timeout = setTimeout(function () { self.timerCallback(); }, 50); }, // Update the SVG button updateButtons: function(play) { document.getElementById("playButton").setAttribute("play", play); document.getElementById("stopButton").setAttribute("play", play); }, // Handling some patterns (text, drawing) has_green_around: function(frameData, pos) { pos_left = pos+ 24; pos_right = pos - 24; r_left = frameData[pos_left+0]; g_left = frameData[pos_left+1]; b_left = frameData[pos_left+2]; r_right = frameData[pos_right+0]; g_right = frameData[pos_right+1]; b_right = frameData[pos_right+2]; return ((r_left < 125 && g_left > 125 && b_left < 80) && (r_right < 125 && g_right > 125 && b_right < 80)); }, is_team_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); //if(hsv[0] < 0.60|| hsv[1] < 0.35) if((hsv[0] < 0.40 || hsv[1] < 0.25 || hsv[2] < 0) || (hsv[0] > 0.70 || hsv[1] > 1 || hsv[2] > 1)) return false; } return true; }, is_ball_color: function(frameData, pos){ for (i=pos-4; i<=pos+4; i+=4){ r = frameData[i+0]; g = frameData[i+1]; b = frameData[i+2]; hsv = rgbToHsv(r,g,b); if((hsv[0] < 0.23 || hsv[1] < 0.40 || hsv[2] < 0.90) || (hsv[0] > 0.27 || hsv[1] > 0.50 || hsv[2] > 1)) return false; } return true; }, dist: function(x1, y1, x2, y2) { return Math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2)); }, computeFrame: function() { try { this.mirrorVideoCtx.clearRect(0, 0, this.width, this.height); this.mirrorVideoCtx.drawImage(this.video, 0, 0, this.width, this.height); } catch(e) { return; } var frame = this.mirrorVideoCtx.getImageData(0, 0, 640, 360); this.mirrorVideoCtx.fillStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.strokeStyle = 'rgba(200,200,200,0.5)'; this.mirrorVideoCtx.lineWidth = 1; var x, y; var weight = 0; var team_shape = null; var ball_shape = null; var ball_found = false; var ball_tracker = document.getElementById("balltracker").checked; var team_tracker = document.getElementById("teamtracker").checked; var path_tracker = document.getElementById("pathtracker").checked; var smart_volume = document.getElementById("smartvolume").checked; var shapes = []; var x, y; var green_bar_left = 0; var green_bar_rigth =0; var green_bar_down = 0; var frame_length = frame.data.length/4; // We dont' need to compute each pixels var step = 4; for (var i = 0; i < frame_length; i += step) { pos = i*4; x = i % this.width; y = Math.round(i / this.width); if (x == 4) green_bar_left += g; else if (x == this.width-4) { green_bar_rigth += g; if (y == 4) green_bar_down += g; } ball_found = ball_tracker && this.is_ball_color(frame.data, pos); team_found = team_tracker && this.is_team_color(frame.data, pos); if (this.has_green_around(frame.data, pos) && (ball_found || team_found)){ if (ball_found) { ball_shape = {}; ball_shape.x = x; ball_shape.y = y; } if(team_found) { if (!team_shape) { // no shape yet, create the first one team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } else { var d = this.dist(x, y, team_shape.x, team_shape.y); if (d>25){ team_shape = {}; team_shape.x = x; team_shape.y = y; team_shape.weight = 1; shapes.push(team_shape); } } } } } if (shapes.length>0) { if (path_tracker){ this.mirrorVideoCtx.beginPath(); this.mirrorVideoCtx.moveTo(shapes[0].x, shapes[0].y); } for( var s=0; s0 && Math.abs(green_left_average-green_right_average)>15){ //this.volumeUp(); this.video.volume=1; document.getElementById("soundIcon").setAttribute("mute", false); }else{ this.volumeDown(); //this.video.volume=0; document.getElementById("soundIcon").setAttribute("mute", true); } } return; } } Source: http://lusob.com/2012/02/tracking-a-football-match-with-html5-and-javascript/

In January, Google released LevelDB, "a fast and lightweight key/value database library." In a recent post on the "High Availability MySQL" blog has generated a discussion around the possibility of LevelDB being a storage engine for MySQL due to its performance benefits. The discussion generated some insight LevelDB's comparative performance to MySQL. The LevelDB site provides some insight into these performance benefits. When creating a brand new database, various methods shows a range of speeds from .4 MB/s to 62.7 MB/s in Write performance. In Read performance, LevelDB ranged from 152 MB/s to 232 MB/s. You can see a more detailed explanation of these benchmarks by checking out the LewisDB site here. The "High Availability MySQL" blog also suggests that LevelDB may be a "great fit" for MongoDB because it does not require multi-statement transactions. Commenters pointed out a few more details about LevelDB that may limit its performance: Unfortunately, there is a trade off between number of SST files and query latency variation: the larger single storage file is - the more time will require to compact it -- Vladmir Rodionov A recent GitHub post also compared MySQL and LevelDB. For sequential insert performance, LevelDB was found to get higher throughput/lower latency overall, although MySQL was more stable. For both average latency and update performance, MySQL and LevelDB performed essentially the same. Have you had a chance to use LevelDB? How does it compare to other libraries? Please post your comments below.

In this post I will demonstrate how to use the propOrder property on the @XmlType annotation to control the ordering of XML elements.

PDF files aren't pleasant. The good news is that they're documented (http://www.adobe.com/devnet/pdf/pdf_reference.html). The bad news is that they're rather complex. I found four Python packages for reading PDF files. http://pybrary.net/pyPdf/ - weak http://www.swftools.org/gfx_tutorial.html - depends on binary XPDF http://blog.didierstevens.com/programs/pdf-tools/ - limited http://www.unixuser.org/~euske/python/pdfminer/ - acceptable I elected to work with PDFMiner for two reasons. (1) Pure Python, (2) Reasonably Complete. This is not, however, much of an endorsement. The implementation (while seemingly correct for my purposes) needs a fair amount of cleanup. Here's one example of remarkably poor programming. # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) Only one of these two is needed; the other is trivially handled as part of the setter method. Also, the package seems to rely on a huge volume of isinstance type checking. It's not clear if proper polymorphism is even possible. But some kind of filter that picked elements by type might be nicer than a lot of isinstance checks. Annotation Extraction While shabby, the good news is that PDFMiner seems to reliably extract the annotations on a PDF form. In a couple of hours, I had this example of how to read a PDF document and collect the data filled into the form. from pdfminer.pdfparser import PDFParser, PDFDocument from pdfminer.psparser import PSLiteral from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter, PDFTextExtractionNotAllowed from pdfminer.pdfdevice import PDFDevice from pdfminer.pdftypes import PDFObjRef from pdfminer.layout import LAParams, LTTextBoxHorizontal from pdfminer.converter import PDFPageAggregator from collections import defaultdict, namedtuple TextBlock= namedtuple("TextBlock", ["x", "y", "text"]) class Parser( object ): """Parse the PDF. 1. Get the annotations into the self.fields dictionary. 2. Get the text into a dictionary of text blocks. The key to the dictionary is page number (1-based). The value in the dictionary is a sequence of items in (-y, x) order. That is approximately top-to-bottom, left-to-right. """ def __init__( self ): self.fields = {} self.text= {} def load( self, open_file ): self.fields = {} self.text= {} # Create a PDF parser object associated with the file object. parser = PDFParser(open_file) # Create a PDF document object that stores the document structure. doc = PDFDocument() # Connect the parser and document objects. parser.set_document(doc) doc.set_parser(parser) # Supply the password for initialization. # (If no password is set, give an empty string.) doc.initialize('') # Check if the document allows text extraction. If not, abort. if not doc.is_extractable: raise PDFTextExtractionNotAllowed # Create a PDF resource manager object that stores shared resources. rsrcmgr = PDFResourceManager() # Set parameters for analysis. laparams = LAParams() # Create a PDF page aggregator object. device = PDFPageAggregator(rsrcmgr, laparams=laparams) # Create a PDF interpreter object. interpreter = PDFPageInterpreter(rsrcmgr, device) # Process each page contained in the document. for pgnum, page in enumerate( doc.get_pages() ): interpreter.process_page(page) if page.annots: self._build_annotations( page ) txt= self._get_text( device ) self.text[pgnum+1]= txt def _build_annotations( self, page ): for annot in page.annots.resolve(): if isinstance( annot, PDFObjRef ): annot= annot.resolve() assert annot['Type'].name == "Annot", repr(annot) if annot['Subtype'].name == "Widget": if annot['FT'].name == "Btn": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'].name elif annot['FT'].name == "Tx": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] elif annot['FT'].name == "Ch": assert annot['T'] not in self.fields self.fields[ annot['T'] ] = annot['V'] # Alternative choices in annot['Opt'] ) else: raise Exception( "Unknown Widget" ) else: raise Exception( "Unknown Annotation" ) def _get_text( self, device ): text= [] layout = device.get_result() for obj in layout: if isinstance( obj, LTTextBoxHorizontal ): if obj.get_text().strip(): text.append( TextBlock(obj.x0, obj.y1, obj.get_text().strip()) ) text.sort( key=lambda row: (-row.y, row.x) ) return text def is_recognized( self ): """Check for Copyright as well as Revision information on each page.""" bottom_page_1 = self.text[1][-3:] bottom_page_2 = self.text[2][-3:] pg1_rev= "Rev 2011.01.17" == bottom_page_1[2].text pg2_rev= "Rev 2011.01.17" == bottom_page_2[0].text return pg1_rev and pg2_rev This gives us a dictionary of field names and values. Essentially transforming the PDF form into the same kind of data that comes from an HTML POST request. An important part is that we don't want much of the background text. Just enough to confirm the version of the form file itself. The cryptic text.sort( key=lambda row: (-row.y, row.x) ) will sort the text blocks into order from top-to-bottom and left-to-right. For the most part, a page footer will show up last. This is not guaranteed, however. In a multi-column layout, the footer can be so close to the bottom of a column that PDFMiner may put the two text blocks together. The other unfortunate part is the extremely long (and opaque) setup required to get the data from the page. Source: http://slott-softwarearchitect.blogspot.com/2012/02/pdf-reading.html

Since EF was released I have been a fan. However, every once in a while I’ll run into a table design situation that I am not sure how to handle with EF. This week, I needed to setup a self-referencing table in order to store some hierarchical data. A self referencing table is a table where the primary key on the table is also defined as a foreign key. Sounds a little confusing right? Let’s clarify the solution with an example. Let’s say I am building an application where I have a list of categories and subcategories. One of my top level categories is “Programming Languages” and under programming languages I have to subcategories which are “C#” and “Java”. In order to store this data I can use a single table with the following structure: The actual data would look like this: Just to clarify, a top level category will have a null value for the ParentId field. For all child categories the ParentId field is used as to represent its parent’s primary key value. As a programmer you may want to think about the ParentId field as a pointer. To complete the example lets take a look at the SQL used to create the table. CREATE TABLE [dbo].[Categories] ( [CategoryId] [int] IDENTITY(1,1) NOT NULL, [Name] [nvarchar](255) NOT NULL, [ParentId] [int] NULL, PRIMARY KEY CLUSTERED ( [CategoryId] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY] GO ALTER TABLE [dbo].[Categories] WITH CHECK ADD CONSTRAINT [Category_Parent] FOREIGN KEY([ParentId]) REFERENCES [dbo].[Categories] ([CategoryId]) GO ALTER TABLE [dbo].[Categories] CHECK CONSTRAINT [Category_Parent] GO Upon examining the SQL, you should have noticed that the CategoryId is the primary key on the table and the ParentId field is a foreign key which points back to the CategoryId field. Since we have a key referencing a another key on the same table we can classify this this as a self-referencing table. Now that we fully understand what a self-referencing table is, we can move forward to the Entity Framework code. To get started we first need to create a simple C# object to represent the Category table. Of course, keep in mind that if you are using EF Code first you do not need to create the table or database ahead of time. I only showed the table first because I wanted to better illustrate what a self referencing table is. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } } So far the Category class is very simple. However, we really want to add a few more properties in order to make this class useful. For example, if you are a child category you really want to be able to use dot notation to get the name of the parent category (e.g. subCategory.Parent.Name). Using EF, we will create a virtual property named Parent. By making the property virtual we are letting EF know that when this property is accessed we want to load some data. Based on your configuration settings and the code you use to retrieve your data (whether or not you used DbSet.Include), EF will lazy load or eager load this data. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } } Finally, we also want a property called Children so we can use dot notation to enumerate over the child categories. Once again, here is the modified class: public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } public virtual ICollection Children { get; set; } } The final step is to let EF know how these properties are related to one another. This can be done using EF's fluent API. If you are new to EF and are unaware of the fluent API then you may want to read this article first. public class CommodityCategoryMap : EntityTypeConfiguration { public CommodityCategoryMap() { HasKey(x => x.CategoryId); Property(x => x.CategoryId) .HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity); Property(x => x.Name) .IsRequired() .HasMaxLength(255); HasOptional(x => x.Parent) .WithMany(x => x.Children) .HasForeignKey(x => x.ParentId) .WillCascadeOnDelete(false); } } Hopefully you paid careful attention to the last section of code where we state the a Category has an optional Parent property. In database speak, this simply means that the ParentID field is nullable. The code also states that if a Category object can have zero or many children. In order to specify that a record is a child, we leverage the ParentId field to hold the primary key value of the parent record. As I mentioned earlier, if you are a programmer its easier to think of the ParentId field as a pointer. Finally, I disabled the cascade on delete option. This step is optional and probably based on your own personal preferences. If you enable cascade on delete and you delete a category that has 100 children then you will effectively remove 101 records. For whatever reason this scares me a little bit. Perhaps, my short career as a DBA caused me to not trust people with large volume delete statements. However, you may decide differently depending on your circumstances. Hopefully, this short EF tutorial will help you if you are working through a scenario where you need to capture and manipulate hierarchical data. If you have any questions please leave a comment.

There is an increasing number of libraries which are described as high performance and have benchmarks to back that claim up. Here is a selection that I am aware of.

XML is for dinosaurs, right? Everybody uses JSON these days. So you do, don’t you? But what about things like XSD, XSLT, JAXB, XPath, etc – is it all evil? In this article, I’d like to introduce the StAXON project (APL2) which tries to give you the best from both worlds: JSON outside, but XML inside. One benefit from this is that you can integrate JSON with powerful XML-related technologies for free. StAXON lets you read and write JSON using the Java Streaming API for XML (javax.xml.stream), also known as StAX. More specifically, StAXON provides implementations of the StAX Cursor API (XMLStreamReader and XMLStreamWriter) StAX Event API (XMLEventReader and XMLEventWriter) StAX Factory API (XMLInputFactory and XMLOutputFactory) for JSON. You may know the Jettison project, which also has XMLStreamReader and XMLStreamWriter implementations. However, StAXON aims to provide a more comprehensive and consistent solution and tries to avoid some of the issues users are having with Jettison. Anyway, let’s get started and see what this “anti-aging substance” for XML can do. Setup Add the following dependency to your Maven POM file: de.odysseus.staxon staxon 1.0 or get the latest StAXON JAR from the Downloads page and add it to your classpath. Mapping Convention The purpose of StAXON’s mapping convention is to generate a more compact JSON. It borrows the "$" syntax for text elements from the Badgerfish convention but attempts to avoid needless text-only JSON objects: Element names become object properties: <–> {"alice":null} Attributes go in properties whose name begin with "@": <–> {"alice":{"@charlie":"david"} Text-only elements go to a simple key/value property: bob <–> {"alice":"bob"} Otherwise, text content is mapped to the "$" property: bob <–> {"alice":{"@charlie":"david","$":"bob"} Nested elements go to nested properties: charlie <–> {"alice":{"bob":"charlie"} A default namespace declaration goes in the element’s "@xmlns" property: <–> {"alice":{"@xmlns":"http://foo.com"} A prefixed namespace declaration goes in the element’s "@xmlns:" property: John Doe555-1111 However, with our JSON-based writer, the output is {"customer":{"name":"John Doe","phone":"555-1111"} Reading JSON Create a JSON-based reader: String json = "{\"customer\":{\"name\":\"John Doe\",\"phone\":\"555-1111\"}"; XMLInputFactory factory = new JsonXMLInputFactory(); XMLStreamReader reader = factory.createXMLStreamReader(new StringReader(json)); Read your document: assert reader.getEventType() == XMLStreamConstants.START_DOCUMENT; reader.nextTag(); assert reader.isStartElement() && "customer".equals(reader.getLocalName()); reader.next(); assert reader.isStartElement() && "name".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "John Doe".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isStartElement() && "phone".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "555-111".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isEndElement(); reader.next(); assert reader.getEventType() == XMLStreamConstants.END_DOCUMENT; reader.close(); Factory Configuration The JsonXMLInputFactory and JsonXMLOutputFactory classes can be configured via the standard setProperty(String, Object) API. The factory classes define several constants for properties they support. However, the JsonXMLConfig interface provides a convenient way to hold the configuration of both - input and output - factories: JsonXMLConfig config = new JsonXMLConfigBuilder(). virtualRoot("customer"). prettyPrint(true). build(); XMLInputFactory inputFactory = new JsonXMLInputFactory(config); ... XMLOutputFactory outputFactory = new JsonXMLOutputFactory(config); ... Virtual Roots Set the virtualRoot configuration property to strip the root element from the JSON representation, e.g. { "name" : "John Doe", "phone" : "555-1111" } As XML requires a single root element, but JSON documents often don’t have one, this is an important feature required to read and write existing JSON formats. Mastering Arrays What about JSON arrays? Unfortunately, there’s nothing like this in XML. And to be honest, this causes most of the trouble when writing JSON via an XML API like StAX. Simply omitting the array boundaries would lead to non-unique JSON properties, which is usually not desired. StAXON provides several ways to deal with JSON arrays. At the core is the idea to leverage XML processing instructions to tell the writer about to start an array: the processing instruction maps a sequence of XML elements with the same name to a JSON array. The processing instruction optionally takes the array element tag name (with prefix) as data. There’s no end array hint as StAXON detects the end of an array sequence and closes it automatically. Consider the following JSON document: { "alice" : { "bob" : [ "edgar", "charlie" ], "peter" : null } } In order to get a "bob" array instead of two separate "bob" properties, we need to provide XML events corresponding to edgar charlie I.e., with the cursor API, you would just insert writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); // to start an array. Initiating Arrays with Element Paths Sometimes it is not desired or even impossible to generate processing instruction to control arrays. This may be the case if the actual writing isn’t done by your code, but some other framework like JAXB or similar, and you only provide a stream writer. Addressing such a scenario, wouldn’t it be nice being able to tell the writer beforehand, which elements should trigger a JSON array? This is where the XMLMultipleStreamWriter and XMLMultipleEventWriter wrappers step in. E.g., to specify a sequence of bob elements below root element alice as a multiple path: writer = new XMLMultipleStreamWriter(writer, true, "/alice/bob"); The boolean parameter specifies whether our paths include the root node (alice) from the paths. That is, we could also use writer = new XMLMultipleStreamWriter(writer, false, "/bob"); To wrap all bob fields into arrays (not just alice children), we can use a relative path, without a leading slash: writer = new XMLMultipleStreamWriter(writer, false, "bob"); Now we (or some legacy code, framework, …) may write our document, and the writer will take care to trigger the bob array for us. Triggering Arrays automatically Finally, if nothing else works for you, you may also let StAXON fully automatically determine array boundaries. Use this only if you cannot provide processing instructions and cannot provide the paths of the elements that should be wrapped into JSON arrays. However, using this method has several drawbacks: The writer basically needs to cache the entire document in memory, eating both space and time. The writer will not be able to produce empty arrays or arrays with a single element. To enable this feature, set the JsonXMLOutputFactory.PROP_AUTO_ARRAY property to true. Triggering Document Arrays StAXON’s writer implementation allows you to wrap a sequence of documents into a JSON array. To do this, write the PI before writing anything else: writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); writer.writeStartDocument(); // first array component ... writer.writeEndDocument(); writer.writeStartDocument(); // second array component ... writer.writeEndDocument(); ... writer.close(); The writer.close() call is crucial here, as it will close the JSON array. Using JAXB Consider a JAXB-annotated Customer class: @JsonXML(virtualRoot = true, prettyPrint = true, multiplePaths = "phone") @XmlRootElement public class Customer { public String name; public List phone; } The @JsonXML annotation is used to configure the mapping details. In the above example, the customer root element is stripped from the JSON representation, phone elements are wrapped into an array and JSON output is nicely formatted, e.g. { "name" : "John Doe", "phone" : [ "555-1111" ] } Now, the JsonXMLMapper class enables for dead-simple mapping to and from JSON: /* * Create mapper instance. */ JsonXMLMapper mapper = new JsonXMLMapper(Customer.class); /* * Read customer. */ InputStream input = getClass().getResourceAsStream("input.json"); Customer customer = mapper.readObject(input); input.close(); /* * Write back to console */ mapper.writeObject(System.out, customer); Using JAX-RS StAXON provides the staxon-jaxrs module, which enables your RESTful services to serialize/deserialize JAXB-annotated classes to/from JSON. It includes the following JAX-RS @Provider classes: de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLObjectProvider is used to read and write JSON objects de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLArrayProvider is used to read and write JSON arrays In order to select the StAXON message body readers/writers for your resource, a @JsonXML annotation is required. When used with JAX-RS, the @JsonXML annotation can be placed on a model type (@XmlRootElement or @XmlType) to configure its serialization and deserialization a JAX-RS resource method to configure serialization of the result type a parameter of a JAX-RS resource method to configure deserialization of the parameter type If a @JsonXML annotation is present at a model type and a resource method or parameter, the latter will override the model type annotation. If neither is present, StAXON will not handle the resource. You can find a sample project using Jersey with StAXON here. Using XPath XPath is another standard that can be easily adopted for use with JSON. The Java XPath API (javax.xml.xpath) doesn’t let us provide an XMLStreamReader or similar as a source, but requires a Document Object Model (DOM). Therefore, we need to read our JSON into a DOM first to apply expressions against that DOM. This could be done by performing an XSLT identity transformation to a DOMResult. However, StAXON provides the DOMEventConsumer class to translate XML events to DOM nodes, which should be faster and simpler than leveraging XSLT. Once we have a DOM, there’s nothing special with applying XPath expressions. StringReader json = new StringReader("{\"edgar\":\"david\",\"bob\":\"charlie\"}"); /* * Our sample JSON has no root element, so specify "alice" as virtual root */ JsonXMLConfig config = new JsonXMLConfigBuilder().virtualRoot("alice").build(); /* * create event reader */ XMLEventReader reader = new JsonXMLInputFactory(config).createXMLEventReader(json); /* * parse JSON into Document Object Model (DOM) */ Document document = DOMEventConsumer.consume(reader); /* * evaluate an XPath expression */ XPath xpath = XPathFactory.newInstance().newXPath(); System.out.println(xpath.evaluate("//alice/bob", document)); Running the above sample will print charlie to the console. What else? In the end, using an XML API to read and write JSON may still look like a compromise, but it may turn out to be a good choice. The availability of a StAX implementation for JSON acts as a door opener to powerful XML related technologies and easily enables for dual-format (XML and JSON) services. There’s more we can do with StAXON: XSD, XSLT, XQuery, XML-JSON/JSON-XML conversions, to name a few. Please check the Wiki for some of those.

Even in languages where there are no constructs but classes, there is no constraint that can force a programmer into writing object-oriented code. In many cases, just wrapping a series of functions into classes do not result in the design. The Convert Procedural Design to Objects has great benefits, but it reaches a very large scale (potentially the whole application). What does object-oriented mean? In 2011, there is no reason to write procedural code anymore in a web application: all libraries and frameworks worth inclusion are object-oriented, even a part of the PHP code (SPL but most importantly PDO, and even DateTime). All other successful languages in the web space are either object-oriented, functional, or both. Software design literature is based on objects and their patterns. However, using class and extends keywords does not suffice to produce an object-oriented design; entire books are written on this topic. This refactoring tries to solve a common case of procedural design shoehorned into an object model: classes containing behavior, and depending on many other ones. dumb classes only being a container for data, or worse primitive types with no methods at all. It is common in procedural design to segregate responsibilities in this procedure/record pattern, but high level methods can be added on these dumb classes to encapsulate a bit of the data they are containing, and simplify the procedural classes using them. It is just a starting point towards "object-orientation", but often an overlooked one. The Tell Don't Ask principle summarizes what we would like to do in very few words: Procedural code gets information then makes decisions. Object-oriented code tells objects to do things. -- Alec Sharp Instead of an infinite series of calls from a procedure to getters and setters, we want to pass messages even to the lower level objects. Steps A preliminary step is to turn primitive data structures into a data object wrapping them and providing getters. If you see variables like arrays or strings passed around in the code to refactor, this step is necessary to provide a class to accomodate potential new methods. Inline the procedural code into a single class. This step makes us able to extract code along different lines than the original ones in the rest of the refactoring: for example, procedural code is often divided in temporal steps, while objects may segregate different parts of the available data instead. Extract methods on the procedural class. See the next steps for hints on what to extract. Methods that have one of the dumb objects as argument can be moved on the object itself, by eliminating it as a parameter but maintaining the remaining ones. Move Method should free the original giant class from any unrelated responsibilities. The goal is to remove logic from the procedural class as much as possibile, going into an opposite direction with regard to the original design; Fowler notes that in some cases the procedural class totally disappears. Example One of my popular examples is invoice calculation: the computation of fields like total price and due taxes from a series of information. In this procedural design, we have one invoice and a bunch of rows modelled with Primitive Obsession (as arrays). assertEquals(4640, $invoice->total()); } } class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $rowTotal = $row[0] + $row[0] * $row[1] / 100; $total += $rowTotal; } return $total; } } We introduce the Row class, but the design is now worse: it adds a bunch of lines of code (the new class) without the new entity giving us something in return. The Row object has no responsibilities, and we just have to write getters and sometimes setters. At least we're writing down parts of our model for documentation (giving names to the net price and tax rate numbers), but we aren't sure this model is the most versatile one. assertEquals(4640, $invoice->total()); } } class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $rowTotal = $row->getNetPrice() + $row->getTaxRate() * $row->getNetPrice() / 100; $total += $rowTotal; } return $total; } } class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function getNetPrice() { return $this->netPrice; } public function getTaxRate() { return $this->taxRate; } } For the scope of this small example all business logic is already in a single class, thus we don't have to inline anything. Let's extract a first method instead: class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $total += $this->rowTotal($row); } return $total; } public function rowTotal($row) { return $row->getNetPrice() + $row->getTaxRate() * $row->getNetPrice() / 100; } } That was a small enough step. In a real situation, the extracted code may be 100-line long, so we would want to test the extraction has been successful before doing anything else. In fact, since the test still passes, we can notice this method has a Row object in its arguments, so it can be moved on Row now that its logic has been clearly isolated: $this->field references should become additional parameters of the method before moving it. Other parameters should just remain formal parameters. Calls to $this->anotherMethod() would be more difficult to treat, as you have the options of moving anothetMethod() in the Row class too, or to extract an interface containing anotherMethod() and pass $this. While moving the code, we change the references to $row to $this, and check that the method scope is public. We also rename the method to total() instead of rowTotal(). { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $total += $row->total(); } return $total; } } class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function getNetPrice() { return $this->netPrice; } public function getTaxRate() { return $this->taxRate; } public function total() { return $this->getNetPrice() + $this->getTaxRate() * $this->getNetPrice() / 100; } } Finally, we inline the getters, since they're not used from outside the Row class. They will be introduced again in the future in case there is a real need for them: as a rule of thumb we avoid exposing any state from Row that is not necessary. class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function total() { return $this->netPrice + $this->taxRate * $this->netPrice / 100; } }

we are entering into the final part of this series, on large scale refactorings : this kind of operations is less predictable and less immediate. however, it is important to be able to perform them with small steps whenever necessary, if we don't want to get stuck in a situation with dozens of broken classes and no clear further step to take. sometimes larger investments are needed to avoid a tangled design: these large scale refactorings use the smaller refactorings as building blocks, but they work at an higher level of abstraction and affect many places in your codebase. why eliminating some inheritance levels? inheritance is easy to abuse , we got it by now. the main problem is its powerful ability to automatically copy and paste code, which leads to use *extends* any time there is duplication instead of any time there is an inexpressed is-a relationship between concepts.. the result is often that you have to jump up and down a hierarchy to follow the thread of execution, disrupting any separation of concerns between subclass and superclass. /the case we want to address today is the combinatorial explosions of concerns where each level of subclassing adds a dimension to the responsibility of the class. consider for example post and link as subclasses of newsfeeditem. at the second level, we may add facebookpost and twitterpost classes, inheriting from post. but also twitterlink and, maybe tomorrow, facebooklink. if we support linkedin, let's think about linkedinpost... the number of classes grow with the square of the elements involved. there are several solutions (like the decorator pattern), but our goal is always the same: keeping a hierarchy as small as possible by allowing a single categorization. when classes of a unique hierarchy can be put in a matrix, they grow with an order of magnitude more than when a single level of inheritance is present. fowler explains how a 3d or 4d matrix is even worse, and require this refactoring to be applied more times to each pair of dimensions. if you have ever seen a pair of withimagefacebookpost and textualfacebookpost classes... steps first, identify the different concerns to separate: all the dimensions you can put the classes on. in our continuing example, we are talking about the socialnetwork categorization and the item one. choose one of the two dimensions to keep in the current hierarchy, while the other will be extracted. perform extract class on the base class of the hierarchy, to introduce a collaborator. this will be the base class of the other hierarchy. introduce subclasses for the extracted object , for each of the original ones that have to be eliminated (so we're talking about only the second categorization.). initialize the current objects with them. move methods onto to the new hierarchy. you may have to extract them first. when subclasses contain only initialization, move the logic in the creation code and eliminate them. this refactoring is a great enabler for further moves: you can extract other collaborators or methods thanks to the reduced complexity of the hierarchies, that can now diverge. you can also simplify testing, by testing at the unit level and for single concerns. example i'll try to keep the logic as small as possible since these examples will use many classes already. in the initial situation, there is a hierarchy with two levels: newsfeeditem defines two abstract methods, the content() and the authorlink(), to use for displaying itself. post and link implements content(), while their subclasses implements authorlink() targeting facebook or twitter respectively. assertequals("enjoy!" . " -- php-cola", $post->__tostring()); } public function testafacebooklinkisdisplayedwithtargetandlinktotheauthor() { $link = new facebooklink("our new ad", "http://youtube.com/...", "php-cola"); $this->assertequals("our new ad" . " -- php-cola", $link->__tostring()); } public function testatwitterlinkisdisplayedwithtargetandlinktotheauthor() { $link = new twitterlink("our new ad", "http://youtube.com/...", "giorgiosironi"); $this->assertequals("our new ad" . " -- @giorgiosironi", $link->__tostring()); } } abstract class newsfeeditem { protected $author; public function __tostring() { return "" . $this->content() . " -- " . $this->authorlink() . ""; } /** * @return string */ protected abstract function content(); /** * @return string */ protected abstract function authorlink(); } abstract class post extends newsfeeditem { private $content; public function __construct($content, $author) { $this->content = $content; $this->author = $author; } protected function content() { return $this->content; } } abstract class link extends newsfeeditem { private $url; private $linktext; public function __construct($linktext, $url, $author) { $this->linktext = $linktext; $this->url = $url; $this->author = $author; } protected function content() { return "url\">$this->linktext"; } } class facebookpost extends post { protected function authorlink() { return "author\">$this->author"; } } class twitterlink extends link { protected function authorlink() { return "author\">@$this->author"; } } class facebooklink extends link { protected function authorlink() { return "author\">$this->author"; } } as you can see, the second level introduces some duplicated code that a composition solution will immediately fix. we choose to maintain post and link in the curent hierarchy, since they are also tied to $this->author. the new hierarchy will contain a facebook and a twitter related class. we add the source new base class for the second hierarchy, and a field in newsfeeditem to hold an instance of it: abstract class newsfeeditem { protected $author; protected $source; public function __tostring() { return "" . $this->content() . " -- " . $this->authorlink() . ""; } /** * @return string */ protected abstract function content(); /** * @return string */ protected abstract function authorlink(); } abstract class source { public function __construct($author) { $this->author = $author; } public abstract function authorlink(); } we add the two facebooksource and twittersource subclasses, and we initialize the $source field to the right instance via a init() hook method. a constructor would be equivalent, but right now we would have to delegate to parent::__construct() and that would be noisy. class facebooksource extends source { public function authorlink() { } } class twittersource extends source { public function authorlink() { } } class facebookpost extends post { public function init() { $this->source = new facebooksource($this->author); } protected function authorlink() { return "author\">$this->author"; } } class twitterlink extends link { public function init() { $this->source = new twittersource($this->author); } protected function authorlink() { return "author\">@$this->author"; } } class facebooklink extends link { public function init() { $this->source = new facebooksource($this->author); } protected function authorlink() { return "author\">$this->author"; } } we perform move method two times to move the authorlink() behavior in the collaborator. this means we have to delegate to $this->source in the base class newsfeeditem. abstract class source { public function __construct($author) { $this->author = $author; } public abstract function authorlink(); } class facebooksource extends source { public function authorlink() { return "author\">$this->author"; } } class twittersource extends source { public function authorlink() { return "author\">@$this->author"; } } now he second level subclasses only contain creation code: we can move eliminate them if we move this initialization in the construction phase, which is represented by the tests here. we can substitute $this->author with $this->source: assertequals("enjoy!" . " -- php-cola", $post->__tostring()); } public function testafacebooklinkisdisplayedwithtargetandlinktotheauthor() { $link = new facebooklink("our new ad", "http://youtube.com/...", new facebooksource("php-cola")); $this->assertequals("our new ad" . " -- php-cola", $link->__tostring()); } public function testatwitterlinkisdisplayedwithtargetandlinktotheauthor() { $link = new twitterlink("our new ad", "http://youtube.com/...", new twittersource("giorgiosironi")); $this->assertequals("our new ad" . " -- @giorgiosironi", $link->__tostring()); } } abstract class newsfeeditem { protected $author; protected $source; public function __tostring() { return "" . $this->content() . " -- " . $this->source->authorlink() . ""; } /** * @return string */ protected abstract function content(); } we can now instantiate directly post and link by making them concrete instead of abstract. you may want to bundle this step with the previous one as it intervene on the same code. a consequence is that we can throw away the second level subclasses. class teaseapartinheritance extends phpunit_framework_testcase { public function testafacebookpostisdisplayedwithtextandlinktotheauthor() { $post = new post("enjoy!", new facebooksource("php-cola")); $this->assertequals("enjoy!" . " -- php-cola", $post->__tostring()); } public function testafacebooklinkisdisplayedwithtargetandlinktotheauthor() { $link = new link("our new ad", "http://youtube.com/...", new facebooksource("php-cola")); $this->assertequals("our new ad" . " -- php-cola", $link->__tostring()); } public function testatwitterlinkisdisplayedwithtargetandlinktotheauthor() { $link = new link("our new ad", "http://youtube.com/...", new twittersource("giorgiosironi")); $this->assertequals("our new ad" . " -- @giorgiosironi", $link->__tostring()); } } the final result can be thought of as a bridge pattern, or just good factoring: a further step could be to divide these tests into unit ones, only exercising a newsfeeditem or a source object. but that's a story for another day...

Right now, when I try to persist an object in Google App Engine, I’m facing the error “Java.lang.VerifyError: Expecting a stackmap frame at branch target“. I’m using JDK 7 and it seems like the problem lies with this JDK. After googling a bit, I found that there seems to be two solutions to fix this problem. Solution 1: Change to JDK 6 As simple as is, change your JDK to version 6 and you won’t be bugged by this exception anymore. Well, in my case, I have to use JDK 7. So, moving on to the solution 2. Solution 2: Configure JVM Go to Windows -> Preferences -> Installed JREs. Select the default JVM and click edit. Then add this parameter as VM argument “-XX:-UseSplitVerifier” as seen below. This should solve the issue. From http://veerasundar.com/blog/2012/01/java-lang-verifyerror-expecting-a-stackmap-frame-at-branch-target-jdk-7/

A short definition of an Android Parcel would be that of a message container for lightweight, high-performance Inter-process communication (IPC). On Android, a "process" is a standard Linux one, and one process cannot normally access the memory of another process, so with Parcels, the Android system decomposes objects into primitives that can be marshaled/unmarshaled across process boundaries. But Parcels can also be used within the same process, to pass data across different components of a same application. As an example, a typical Android application has several screens, called "Activities" , and needs to communicate data or action from one Activity to the next. To write an object than can be passed through, we can implement the Parcelable interface. Android itself provides a built-in Parcelable object called an Intent which is used to pass information from one component to another. Using an Intent is pretty straightforward. Let's say we're collecting user data from our initial screen called CollectDataActivity. // inside CollectDataActivity, construct intent to pass along the next Activity, i.e. screen Intent in = new Intent(this, ProcessDataActivity.class); in.putExtra("userid", id); // (key,value) pairs in.putExtra("age", age); in.putExtra("phone", phone); in.putExtra("is_registered", true); // call next Activity --> next screen comes up startActivity(in); We need to collect that information from our data collection screen to process it. So all we do is the following: // inside ProcessDataActivity, get the info needed from previous Activity Intent in = this.getIntent(); in.getLongExtra("userid", 0L); in.getIntExtra("age", 0); in.getStringExtra("phone"); in.getBooleanExtra("is_registered", false); // false = default value overridden by user input Again, pretty straightforward. We retrieve the data using the same keys used to send it, and using our Intent's corresponding methods for each data type. But even when communicating with Intents, we can still use Parcels to pass data within the intent. For instance, we can do the above in a more elegant way using a custom, Parcelable User class: In the first Activity: // in CollectDataActivity, populate the Parcelable User object using its setter methods User usr = new User(); usr.setId(id); // collected from user input// etc.. // pass it to another component Intent in = new Intent(this, ProcessDataActivity.class); in.putExtra("user", usr); startActivity(in); In the second Activity: // in ProcessDataActivity retrieve User Intent intent = getIntent(); User usr = (User) intent.getParcelableExtra("user"); And this is what a Parcelable User class looks like: import android.os.Parcel; import android.os.Parcelable; public class User implements Parcelable { private long id; private int age; private String phone; private boolean registered; // No-arg Ctor public User(){} // all getters and setters go here //... /** Used to give additional hints on how to process the received parcel.*/ @Override public int describeContents() { // ignore for now return 0; } @Override public void writeToParcel(Parcel pc, int flags) { pc.writeLong(id); pc.writeInt(age); pc.writeString(phone); pc.writeInt( registered ? 1 :0 ); } /** Static field used to regenerate object, individually or as arrays */ public static final Parcelable.Creator CREATOR = new Parcelable.Creator() { public User createFromParcel(Parcel pc) { return new User(pc); } public User[] newArray(int size) { return new User[size]; } }; /**Ctor from Parcel, reads back fields IN THE ORDER they were written */ public User(Parcel pc){ id = pc.readLong(); age = pc.readInt(); phone = pc.readString(); registered = ( pc.readInt() == 1 ); } } What we did was: Make our User class implement the Parcelable interface. Parcelable is not a marker interface, hence what follows: Implement its describeContents method, which in this case does nothing. Implement its abstract method writeToParcel, which takes the current state of the object and writes it to a Parcel Add a static field called CREATOR to our class, which is an object implementing the Parcelable.Creator interface Add a Constructor that takes a Parcel as parameter. The CREATOR calls that constructor to rebuild our object. This looks like a lot of extra code at first, but bear in mind that, as in most cases, our application might evolve into incorporating more data from the user... Sometimes we need to pass complex objects from one component to another, and passing an object yields a cleaner design. The same logic applies for communicating between an Activity (foreground UI) and a background Service. We would just call the startService method instead of startActivity and pass it our Parcelable User object. Note that a Service is not running in a separate process by default. At this point, there are a couple of questions that may be raised: Isn't using an IPC-friendly, custom object for in-process communication simply overkill? Why would we want to use Parcelable, when we already have built-in Java serialization? The answer to the first concern is...maybe. But communicating through a custom object than through a list of key-value pairs is more OO, and it has no noticeable negative performance impact. As for the second question, why not simply have User implement Serializable, a theoretically simpler, marker interface? In one word, performance. Using Parcels is more efficient than serializing, at the price of some added complexity. That extra efficiency has in turn its limits: passing an image ( Bitmap) using Parcelable is generally not a good idea (although Bitmap does in fact implement Parcelable). A much more memory-efficient way would be to pass only its URI or Resource ID, so that other Android components in your application can have access to it. Another limitation of Parcelable is that it must not be used for general-purpose serialization to storage, since the underlying implementation may vary with different versions of the Android OS. So yes, Parcels are faster by design, but as high-performance transport, not as a replacement for general-purpose serialization mechanism. Having said all that, since our User object is Parcelable, it can now be sent from this application to another one running in another process, in particular through an interface implementing a remote service. In an upcoming post, we'll look at IPC and Android's Interface Definition Language (AIDL). from Tony's Blog

i want to share something a software developer or operations guy every now and then encounters. and which i did yesterday. it all started with our nightly maven build failing last saturday jan 28 on a java.lang.noclassdeffounderror: javax/xml/bind/validationeventlocator the interesting snippet from the concole output from this build looked like this: [info] [jaxb2:generate {execution: xxx-schema-gen}] [fatal error] org.jvnet.mjiip.v_2.xjc2mojo#execute() caused a linkage error (java.lang.noclassdeffounderror) and may be out-of-date. check the realms: [fatal error] plugin realm = app0.child-container[org.jvnet.jaxb2.maven2:maven-jaxb2-plugin] urls[0] = file:/home/hudson/.m2/repository/org/jvnet/jaxb2/maven2/maven-jaxb2-plugin/0.8.1/maven-jaxb2-plugin-0.8.1.jar urls[1] = file:/home/hudson/.m2/repository/org/jvnet/jaxb2/maven2/maven-jaxb2-plugin-core/0.8.1/maven-jaxb2-plugin-core-0.8.1.jar urls[2] = file:/home/hudson/.m2/repository/com/sun/org/apache/xml/internal/resolver/20050927/resolver-20050927.jar urls[3] = file:/home/hudson/.m2/repository/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar urls[4] = file:/home/hudson/.m2/repository/org/codehaus/plexus/plexus-utils/1.5.15/plexus-utils-1.5.15.jar urls[5] = file:/home/hudson/.m2/repository/org/jfrog/maven/annomojo/maven-plugin-anno/1.3.1/maven-plugin-anno-1.3.1.jar urls[6] = file:/home/hudson/.m2/repository/org/jvnet/jaxb2/maven2/maven-jaxb22-plugin/0.8.1/maven-jaxb22-plugin-0.8.1.jar urls[7] = file:/home/hudson/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.5-b10/jaxb-impl-2.2.5-b10.jar urls[8] = file:/home/hudson/.m2/repository/com/sun/xml/bind/jaxb-xjc/2.2.5-b10/jaxb-xjc-2.2.5-b10.jar [fatal error] container realm = plexus.core.maven [info] ------------------------------------------------------------------------ [error] fatal error [info] ------------------------------------------------------------------------ [info] javax/xml/bind/validationeventlocator [info] ------------------------------------------------------------------------ [info] trace java.lang.noclassdeffounderror: javax/xml/bind/validationeventlocator at com.sun.tools.xjc.reader.internalizer.domforestscanner.scan(domforestscanner.java:84) it seems that the maven-jaxb2-plugin –we use for jaxb to generate java sources for a certain schema–suddenly triggers a (missing or incompatible?) dependency. the weird thing is the “suddenly” part, since the last commit was done on friday after which the normal continuous builds ran fine and the even the nightly build on midnight friday to saturday. we use maven to take care of these things right? always have a known set of dependencies, each part of the process? ofcourse at the time things didn’t seem so obvious. first thing i wondered, could it have something do to with a java 5 vs 6 problem? as a matter of fact, on our project we’re trying to go to java 6 after a long time, but i’m the only one which already upgraded my local developer machine to work with jdk6 for a while and see if nothing strange happens. you know, keeping a lot of files from your workspace in a special “do not commit” working set or hidden from view, in order to prevent it from prematurely ending up in version control so naturally, i first explored all changesets leading up to the failed build to see if i by accident committed a file i shouldn’t have. but i couldn’t find anything. next i thought: could hudson be configured incorrectly last week by a parting co-worker (which had a somewhat unofficial ownership of our ci infrastructure until he left) which now surfaced, because e.g. a server was restarted or hudson was upgraded? checked a few things out: our hudson startup.sh looked like /usr/java/latest/bin/java -jar hudson.war --httpport=9090 --ajp13port=9091 --deamon --logfile=hudson.log & where /usr/java/latest/bin/java pointed to “java(tm) se runtime environment (build 1.6.0 _24-b07)” while normal “java” on the command-line pointed to “java(tm) 2 runtime environment, standard edition (build 1.5.0 _22-b03)” . could it be that the “latest” directory pointed earlier to java 5 en (by some mysterious upgrade on the server) now suddenly links to java 6. could the java version we start hudson with affect the java version we use for our hudson jobs? i don’t hope so! consequently, i checked the hudson configuration itself: what kind of jdk’s did we configure? luckily, only one: named “1_5_0″ with java_home pointing to “/usr/java/default”. but again not clear from the path to what java version it points, until i figured out this symlink leads to “/usr/java/jdk1.5.0_22″. good, jdk 5 should be used as default for all our maven builds. to verify i added a 2nd bogus jdk entry in the hudson configuration, so i was able to explicitly choose it in the job configuration – and after triggering a new build i was sure hudson was configured correctly jdk-wise. man, how one can digress! so let’s take a look at the offending library at hand. from the stacktrace one can spot the maven-jaxb2-plugin artefact from the group org.jvnet.jaxb2.maven2 . did something happen to this dependency itself? i opened up our nexus console, searched for it and saw the screen below: wait a minute! why do we have two versions of this plugin? in nexus i opened up the tab artefact information on the 0.8.1 version and saw that the uploaded date listed last saturday 28th of jan. i know we shouldn’t have more than one version of this plugin in nexus. did a small check on the jira release notes page of this plugin… well then. so a new release of the plugin was published online last saturday and the nightly build triggered the download of a new version 0.8.1 – which seemed to be compiled against java 6 now which caused the java.lang.noclassdeffounderror . this naturally lead me to the offending part in our pom.xml : org.jvnet.jaxb2.maven2 maven-jaxb2-plugin argh. no version! changed it into: org.jvnet.jaxb2.maven2 maven-jaxb2-plugin 0.8.0 i committed the pom.xml and triggerd a manual build – and… it worked. ofcourse. in conclusion there are some lessons to be learned i think: when debugging a problem, always try to reason about possible paths which come to mind to follow first and don’t get hung on your very first idea which pops up. this got me i think sidetracked too long on finding out whether or not i accidentally committed java 6 files or whether or not hudson got accidentally misconfigured into not using java 5 anymore somehow. troubleshooting skills are highly dependent of previous experience in a field and ofcourse, if ever a similar problem arises i’d be able to recognize it faster now. write it down – so a collegue or yourself can find it back more easily later on. explicity set dependency versions in your pom.xml! don’t blame maven for everything does anyone else have a troubleshooting-tale of going in the wrong direction? original posting: http://tedvinke.wordpress.com/2012/02/01/why-does-my-maven-build-fail

While Kotlin programming language prepares for the public early access program I want to share with you one an example how easly it can be used with existing Java codebases. This short note does not intend to teach the reader how to use Kotlin but only to make him/her intersted to learn. We will create small toy application using Kotlin and the charming Guice framework. The application is nothing more than the usual "Hello, World!" but it contains an application service and logging service and each service will have two incarnations - for testing and for production use. Let us start with an abstract LoggingService. It will be an abstract class with only one abstract method to output the passed string. Most probably the code below does not require much explaination. abstract class LoggingService() { abstract fun info(message: String?) } Now we will define a simple implementation, which prints the given message to the standard output class StdoutLoggingService() : LoggingService(){ override fun info(message: String?) = println("INFO: $message") } override modifier tells the compiler that the method overrides some method of a superclass. Note also string interpolation Now we will create a little more sophisticated implementation, which we will use in production :) class JdkLoggingService [Inject] (protected val jdkLogger: Logger) : LoggingService() { override fun info(message: String?) = jdkLogger.info(message) } [Inject] annotation on constructor tells Guice how to create JdkLoggerService. val on constructor parameter asks to create final property initialized by given value. var asks for non-final one and neither of them define simple constructor parameter Now we are ready to define the abstract application service and both implementations of it for test and production use. abstract class AppService(protected val logger: LoggingService, protected val mode: String) { fun run() { logger.info("Application started in '$mode' mode with ${logger.javaClass.getName()} logger") logger.info("Hello, World!") } } class RealAppService [Inject] (logger: LoggingService) : AppService(logger, "real") class TestAppService [Inject] (logger: LoggingService) : AppService(logger, "test") Note how elegantly Kotlin allows us to define properties and pass values to superconstructor Now we go to more interesting part - creating min-DSL for Guice. But let us start with how we want our application to look like. fun main(args: Array) { fun configureInjections(test: Boolean) = injector { +module { if(test) { bind().toInstance (StdoutLoggingService()) bind().to () } else { bind().to() bind().to () } } } configureInjections(test=false).getInstance ().run() } There are few things to notice here We use local function to configure injections. It is not necessary but allow some nice code grouping. Return type of function configureInjections is inferred from it's body (call to finction injector - root of our DSL which we will define soon) We call configureInjections using named arguments syntax. It is not necessary in our case but very convinient when we have many parameters and some of them are optional. Now it is time to define simple parts of our DSL. It will be three extension functions for some Guice classes. fun Binder.bind() = bind(javaClass()).sure() fun AnnotatedBindingBuilder.to() = to(javaClass()).sure() fun Injector.getInstance() = getInstance(javaClass()) Please read about extension functions in Kotlin documentation Note in variance used in function 'to'. Please read about generics and variance in Kotlin documentation. It is really interesting javaClass() call is Kotlin standard library for Java (remember that Kotlin can be compiled for other platforms like javascript as well) is equivalent to T.class in Java. The huge difference here is that in Kotlin it applicable not only to real classes but also to generic type parameters (thanks to runtime type information). Call to method sure() is Kotlin way (soon to be replaced by special language construct) to ensure non-nullability. As Java has not notation of nullable and non-nullable types we need to take some care on integration point. Please read amazing story of null-safety in Kotlin documentation. Finally we are ready to most complex part - definition of method injector. fun injector(config: ModuleCollector.() -> Any?) : Injector { val collector = ModuleCollector() collector.config() return Guice.createInjector(collector.collected).sure() } It has one parameter of type ModuleCollector.()->Any? which means "extension function for ModuleCollector(we will define ModuleCollector few lines below), which does not accept any parameters and returns value of any type potentially nullable" The implementation is very straight-forward we create ModuleCollector we use it as receiver to call given configuration function we ask Guice to create Injector for all modules configured (consult Guice documentation for details) This is extremely powerful method of defining DSLs in Kotlin As we saw above we call method injector with function block expression. The fact that parameter is extension function makes all (modulo visibility rules) methods of ModuleCollector available inside function block. The last part of our DSL is definition of ModuleCollector. It contains internal list of collected modules and only two methods method module - uses variation of the same extension-function-as-parameter trick - we use extension function to implement method of anonimous class. Please consult Kotlin documentation on object expressions method plus - overrides unary plus operation Please consult Kotlin documentation on operator overloading. It is very important to note that this method is both extension method and member of class ModuleCollector. It allows very good context catching on call site. In our case above we call this method inside extention function with ModuleCollectoror receiver and apply it to Module created by call to method Module. class ModuleCollector() { val collected = ArrayList () fun module (config: Binder.()->Any?) : Module = object: Module { override fun configure(binder: Binder?) { binder.sure().config() } } fun Module.plus() { collected.add(this) } } We are done. I hope that was interesting. This note is a very brief and non-detailed introduction on goodies of Kotlin. If this has motivated you to read more about Kotlin then my goal is achieved. Thank you and till next time!

Another mock recipe, this one for mocking generator methods. A Python generator is a function or method that uses the yield statement to return a series of values when iterated over (there are also generator expressions and more advanced uses of generators, but we aren't concerned about them here. A very good introduction to generators and how powerful they are is: Generator Tricks for Systems Programmers.). A generator method / function is called to return the generator object. It is the generator object that is then iterated over. The protocol method for iteration is __iter__, so we can mock this using a MagicMock. Here's an example class with an "iter" method implemented as a generator: >>> class Foo(object): ... def iter(self): ... for i in [1, 2, 3]: ... yield i ... >>> foo = Foo() >>> list(foo.iter()) [1, 2, 3] How would we mock this class, and in particular its "iter" method? To configure the values returned from the iteration (implicit in the call to list), we need to configure the iterator returned by the call to foo.iter(). >>> values = [1, 2, 3] >>> mock_foo = MagicMock() >>> iterable = mock_foo.iter.return_value >>> iterator = iter(values) >>> iterable.__iter__.return_value = iterator >>> list(mock_foo.iter()) [1, 2, 3] The above example is done step-by-step. The shorter version is: >>> mock_foo = MagicMock() >>> mock_foo.iter.return_value.__iter__.return_value = iter([1, 2, 3]) >>> list(mock_foo.iter()) [1, 2, 3] This is now also in the docs on the mock examples page. There's now quite a collection of useful mock recipes there, so even if you're an experienced mock user it is worth a browse. Source: http://www.voidspace.org.uk/python/weblog/arch_d7_2011_06_11.shtml

Previously, we covered some tips and tricks for the Grid control. In this article, we will go over a few tips and tricks for the wx.ListCtrl widget when it’s in “report” mode. Take a look at the tips below: How to create a simple ListCtrl How to sort the rows of a ListCtrl How to make the ListCtrl cells editable in place Associating objects with ListCtrl rows Alternate the row colors of a ListCtrl How to create a simple ListCtrl The list control is a pretty common widget. In Windows, you will see the list control in Windows Explorer. It has four modes: icon, small icon, list, and report. They roughly match up with icons, tiles, list, and details views in Windows Explorer respectively. We’re going to focus on the ListCtrl in Report mode because that’s the mode that most developers use it in. Here’s a simple example of how to create a list control: import wx ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = wx.Panel(self, wx.ID_ANY) self.index = 0 self.list_ctrl = wx.ListCtrl(panel, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.InsertColumn(0, 'Subject') self.list_ctrl.InsertColumn(1, 'Due') self.list_ctrl.InsertColumn(2, 'Location', width=125) btn = wx.Button(panel, label="Add Line") btn.Bind(wx.EVT_BUTTON, self.add_line) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL|wx.CENTER, 5) panel.SetSizer(sizer) #---------------------------------------------------------------------- def add_line(self, event): line = "Line %s" % self.index self.list_ctrl.InsertStringItem(self.index, line) self.list_ctrl.SetStringItem(self.index, 1, "01/19/2010") self.list_ctrl.SetStringItem(self.index, 2, "USA") self.index += 1 #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() As you can probably tell from the code above, it’s really easy to create a ListCtrl instance. Notice that we set the style to report mode using the wx.LC_REPORT flag. To add column headers, we call the ListCtrl’s InsertColumn method and pass an integer to tell the ListCtrl which column is which and a string for the user’s convenience. Yes, the columns are zero-based, so the first column is number zero, the second column is number one, etc. The next important piece is contained in the button’s event handler, add_line, where we learn how to add rows of data to the ListCtrl. The typical method to use is the InsertStringItem method. If you wanted an image added to each row as well, then you’d use a more complicated method like InsertColumnInfo along with the InsertImageStringItem method. You can see how to use them in the wxPython demo. We’re sticking with the easy stuff in this article. Anyway, when you call InsertStringItem you give it the correct row index and a string. You use the SetStringItem method to set the data for the other columns of the row. Notice that the SetStringItem method requires three parameters: the row index, the column index and a string. Lastly, we increment the row index so we don’t overwrite anything. Now you can get out there and make your own! Let’s continue and find out how to sort rows! How to sort the rows of a ListCtrl The ListCtrl widget has had some extra scripts written for it that add functionality to the widget. These scripts are called mixins. You can read about them here. For this recipe, we’ll be using the ColumnSorterMixin mixin. The code below is a stripped down version of one of the wxPython demo examples. import wx import wx.lib.mixins.listctrl as listmix musicdata = { 0 : ("Bad English", "The Price Of Love", "Rock"), 1 : ("DNA featuring Suzanne Vega", "Tom's Diner", "Rock"), 2 : ("George Michael", "Praying For Time", "Rock"), 3 : ("Gloria Estefan", "Here We Are", "Rock"), 4 : ("Linda Ronstadt", "Don't Know Much", "Rock"), 5 : ("Michael Bolton", "How Am I Supposed To Live Without You", "Blues"), 6 : ("Paul Young", "Oh Girl", "Rock"), } ######################################################################## class TestListCtrl(wx.ListCtrl): #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): wx.ListCtrl.__init__(self, parent, ID, pos, size, style) ######################################################################## class TestListCtrlPanel(wx.Panel, listmix.ColumnSorterMixin): #---------------------------------------------------------------------- def __init__(self, parent): wx.Panel.__init__(self, parent, -1, style=wx.WANTS_CHARS) self.index = 0 self.list_ctrl = TestListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN |wx.LC_SORT_ASCENDING ) self.list_ctrl.InsertColumn(0, "Artist") self.list_ctrl.InsertColumn(1, "Title", wx.LIST_FORMAT_RIGHT) self.list_ctrl.InsertColumn(2, "Genre") items = musicdata.items() index = 0 for key, data in items: self.list_ctrl.InsertStringItem(index, data[0]) self.list_ctrl.SetStringItem(index, 1, data[1]) self.list_ctrl.SetStringItem(index, 2, data[2]) self.list_ctrl.SetItemData(index, key) index += 1 # Now that the list exists we can init the other base class, # see wx/lib/mixins/listctrl.py self.itemDataMap = musicdata listmix.ColumnSorterMixin.__init__(self, 3) self.Bind(wx.EVT_LIST_COL_CLICK, self.OnColClick, self.list_ctrl) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- # Used by the ColumnSorterMixin, see wx/lib/mixins/listctrl.py def GetListCtrl(self): return self.list_ctrl #---------------------------------------------------------------------- def OnColClick(self, event): print "column clicked" event.Skip() ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = TestListCtrlPanel(self) #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() This code is a little on the odd side in that we have inherit the mixin in the wx.Panel based class rather than the wx.ListCtrl class. You can do it either way though as long as you rearrange the code correctly. Anyway, we are going to home in on the key differences between this example and the previous one. The first difference of major importance is in the looping construct where we insert the list control’s data. Here we include the list control’s SetItemData method to include the necessary inner-workings that allow the sorting to take place. As you might have guessed, this method associates the row index with the music data dict’s key. Next we instantiate the ColumnSorterMixin and tell it how many columns there are in the list control. We could have left the EVT_LIST_COL_CLICK binding off this example as it has nothing to do with the actual sorting of the rows, but in the interest of increasing your knowledge, it was left in. All it does is show you how to catch the user’s column click event. The rest of the code is self-explanatory. If you want to know about the requirements for this mixin, especially when you have images in your rows, please see the relevant section in the source (i.e. listctrl.py). Now, wasn’t that easy? Let’s continue our journey and find out how to make the cells editable! How to make the ListCtrl cells editable in place Sometimes, the programmer will want to allow the user to click on a cell and edit it in place. This is kind of a lightweight version of the wx.grid.Grid control. Here’s an example: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class EditableListCtrl(wx.ListCtrl, listmix.TextEditMixin): ''' TextEditMixin allows any column to be edited. ''' #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): """Constructor""" wx.ListCtrl.__init__(self, parent, ID, pos, size, style) listmix.TextEditMixin.__init__(self) ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = EditableListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "Editable List Control") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this script, we put the TextEditMixin in our wx.ListCtrl class instead of our wx.Panel, which is the opposite of the previous example. The mixin itself does all the heavy lifting. Again, you’ll have to check out the mixin’s source to really understand how it works. Associating objects with ListCtrl rows This subject comes up a lot: How do I associate data (i.e. objects) with my ListCtrl’s rows? Well, we’re going to find out exactly how to do that with the following code: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.myRowDict[index] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[currentItem] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The list control widget actually doesn’t have a built-in way to accomplish this feat. If you want that, then you’ll want to check out the ObjectListView widget, which wraps the ListCtrl and gives it a lot more functionality. In the meantime, we’ll take a minute and go over the code above. The first piece is just a plain Car class with four attributes. Then in the MyPanel class, we create a list of Car objects that we’ll use for the ListCtrl’s data. To add the data to the ListCtrl, we use a for loop to iterate over the list. We also associate each row with a Car object using a Python dictionary. We use the row’s index for the key and the dict’s value ends up being the Car object. This allows us to access all the Car/row object’s data later on in the onItemSelected method. Let’s check that out! In onItemSelected, we grab the row’s index with the following little trick: event.m_itemIndex. Then we use that value as the key for our dictionary so that we can gain access to the Car object associated with that row. At this point, we just print out all the Car object’s attributes, but you could do whatever you want here. This basic idea could easily be extended to use a result set from a SqlAlchemy query for the ListCtrl’s data. Hopefully you get the general idea. Now if you were paying close attention, like Robin Dunn (creator of wxPython) was, then you might notice some really silly logic errors in this code. Did you find them? Well, you won’t see it unless you sort the rows, delete a row or insert a row. Do you see it now? Yes, I stupidly based the “unique” key in my dictionary on the row’s position, which will change if any of those events happen. So let’s look at a better example: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.id = id(self) self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.list_ctrl.SetItemData(index, row.id) self.myRowDict[row.id] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this example, we add a new attribute to our Car class that creates a unique id for each instance that is created using Python’s handy id builtin. Then in the loop where we add the data to the list control, we call the widget’s SetItemData method and give it the row index and the car instance’s unique id. Now it doesn’t matter where the row ends up because it’s had the unique id affixed to it. Finally, we have to modify the onItemSelected to get the right object. The magic happens in this code: # this code was helpfully provided by Robin Dunn car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] Cool, huh? Our last example will cover how to alternate the row colors, so let’s take a look! Alternate the row colors of a ListCtrl As this section’s title suggests, we will look at how to alternate colors of the rows of a ListCtrl. Here’s the code: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) if index % 2: self.list_ctrl.SetItemBackgroundColour(index, "white") else: self.list_ctrl.SetItemBackgroundColour(index, "yellow") index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control w/ Alternate Colors") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The code above will alternate each row’s background color. Thus you should see yellow and white rows. We do this by calling the ListCtrl instance’s SetItemBackgroundColour method. If you were using a virtual list control, then you’d want to override the OnGetItemAttr method. To see an example of the latter method, open up your copy of the wxPython demo; there’s one in there. Wrapping Up We’ve covered a lot of ground here. You should now be able to do a lot more with your wx.ListCtrl than when you started, assuming you’re new to using it, of course. Feel free to ask questions in the comments or suggest future recipes. I hope you found this helpful! Note: All examples were tested on Windows XP with Python 2.5 and wxPython 2.8.10.1. They were also tested on Windows 7 Professional with Python 2.6 Additional Reading The official wxPython wx.ListCtrl documentation The ListControls wiki page ListCtrl Tooltips wiki page The ObjectListView website The UltimateListCtrl, a pure Python implementation now included with wxPython Source Code listctrl.zip listctrl.tar Source: http://www.blog.pythonlibrary.org/2011/01/04/wxpython-wx-listctrl-tips-and-tricks/

The simplest in-memory cache implementation should support Addition of objects into cache either via key-value, or via object creation mechanism Deletion of objects from cache based on key, or object type Querying cache store to check existence of an object There are several ways to achieve this using multiple design patterns. But if we were to implement those design patterns in our applications, we would end up designing a framework similar to Enterprise Library Caching block. So to keep things fairly simple – we need a simple implementation of caching objects in-memory and this cache to be thread-safe for multi-threading applications. So for that, you can just copy this piece of code into your application and you should be all set with an in-memory cache. public static class CacheStore { /// /// In-memory cache dictionary /// private static Dictionary _cache; private static object _sync; /// /// Cache initializer /// static CacheStore() { _cache = new Dictionary(); _sync = new object(); } /// /// Check if an object exists in cache /// /// Type of object /// Name of key in cache /// True, if yes; False, otherwise public static bool Exists(string key) where T : class { Type type = typeof(T); lock (_sync) { return _cache.ContainsKey(type.Name + key); } } /// /// Check if an object exists in cache /// /// Type of object /// True, if yes; False, otherwise public static bool Exists() where T : class { Type type = typeof(T); lock (_sync) { return _cache.ContainsKey(type.Name); } } /// /// Get an object from cache /// /// Type of object /// Object from cache public static T Get() where T : class { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(type.Name) == false) throw new ApplicationException("An object of the desired type does not exist: " + type.Name); lock (_sync) { return (T)_cache[type.Name]; } } } /// /// Get an object from cache /// /// Type of object /// Name of key in cache /// Object from cache public static T Get(string key) where T : class { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(key + type.Name) == false) throw new ApplicationException(String.Format("An object with key '{0}' does not exists", key)); lock (_sync) { return (T)_cache[key + type.Name]; } } } /// /// Create default instance of the object and add it in cache /// /// Class whose object is to be created /// Object of the class public static T Create(string key, params object[] constructorParameters) where T : class { Type type = typeof(T); T value = (T)Activator.CreateInstance(type, constructorParameters); lock (_sync) { if (_cache.ContainsKey(key + type.Name)) throw new ApplicationException(String.Format("An object with key '{0}' already exists", key)); lock (_sync) { _cache.Add(key + type.Name, value); } } return value; } /// /// Create default instance of the object and add it in cache /// /// Class whose object is to be created /// Object of the class public static T Create(params object[] constructorParameters) where T : class { Type type = typeof(T); T value = (T)Activator.CreateInstance(type, constructorParameters); lock (_sync) { if (_cache.ContainsKey(type.Name)) throw new ApplicationException(String.Format("An object of type '{0}' already exists", type.Name)); lock (_sync) { _cache.Add(type.Name, value); } } return value; } public static void Add(string key, T value) { Type type = typeof(T); if (value.GetType() != type) throw new ApplicationException(String.Format("The type of value passed to cache {0} does not match the cache type {1} for key {2}", value.GetType().FullName, type.FullName, key)); lock (_sync) { if (_cache.ContainsKey(key + type.Name)) throw new ApplicationException(String.Format("An object with key '{0}' already exists", key)); lock (_sync) { _cache.Add(key + type.Name, value); } } } /// /// Remove an object type from cache /// /// Type of object public void Remove() { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(type.Name) == false) throw new ApplicationException(String.Format("An object of type '{0}' does not exists in cache", type.Name)); lock (_sync) { _cache.Remove(type.Name); } } } /// /// Remove an object stored with a key from cache /// /// Type of object /// Key of the object public void Remove(string key) { Type type = typeof(T); lock (_sync) { if (_cache.ContainsKey(key + type.Name) == false) throw new ApplicationException(String.Format("An object with key '{0}' does not exists in cache", key)); lock (_sync) { _cache.Remove(key + type.Name); } } } } Every method has 2 overloads With Key as a parameter: This method adds a new key-value in the cache store for a particular object type. This also means that for a particular object (say Employee), you can have multiple cached-objects (say, multiple employees in an organization) Without Key as a parameter – This method adds a new key (type of the object) and value in the cache store. This means, for a particular object type (say ConfigurationSettings) there will single object in the cache (say, configuration value) Implementation example using CacheStore is: MonoAssemblyResolver targetAssembly = null; if (CacheStore.Exists(projMapping.TargetAssemblyPath)) { targetAssembly = CacheStore.Get(projMapping.TargetAssemblyPath); } else { targetAssembly = new MonoAssemblyResolver(projMapping.TargetAssemblyPath); CacheStore.Add(projMapping.TargetAssemblyPath, targetAssembly); } Since this uses plain-C# and is light weight, this can be used in ASP.NET MVC, Silverlight, WPF, or Windows Phone applications. So happy coding! Source: http://www.ganshani.com/2012/01/31/in-memory-cache-implementation-in-c