In Eclipse, when you press Ctrl button and click on any Class names, the IDE will take you to the source file for that class. This is the normal behavior for the classes you have in your project. But, in case you want the same behavior for Java’s core classes too, you can have it by attaching the Java source with the Eclipse IDE. Once you attach the source, thereafter when you Ctrl+Click any Java class names (String for example), Eclipse will open the source code of that class. To attach the Java source code with Eclipse, When you install the JDK, you must have selected the option to install the Java source files too. This will copy the src.zip file in the installation directory. In Eclipse, go to Window -> Preferences -> Java -> Installed JREs -> Add and choose the JDK you have in your system. Eclipse will now list the JARs found in the dialog box. There, select the rt.jar and choose Source Attachment. By default, this will be pointing to the correct src.zip. If not, choose the src.zip file which you have in your java installation directory. Similarly, if you have the javadoc downloaded in your machine, you can configure that too in this dialog box. Done! Here after, for all the projects for which you are using the above JDK, you’ll be able to browse the Java’s source code just like how you browse your own code. From http://veerasundar.com/blog/2011/08/attaching-java-source-with-eclipse-ide

JSDoc is the de facto standard for documenting JavaScript code. You need to know at least its syntax (which is also used by many other tools) if you publish code. Alas, documentation is still scarce, but this post can help – it shows you how to run JSDoc and how its syntax works. (The JSDoc wiki [2] is the main source of this post, some examples are borrowed from it.) As a tool, JSDoc takes JavaScript code with special /** */ comments and produces HTML documentation for it. For example: Given the following code. /** @namespace */ var util = { /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ repeat: function(str, times) { if (times === undefined || times < 1) { times = 1; } return new Array(times+1).join(str); } }; The generated HTML looks as follows in a web browser: This post begins with a quick start, so can try out JSDoc immediately if you are impatient. Afterwards, more background information is given. 1. Quick start For the steps described below, you need to have Java installed. JSDoc includes the shell script jsrun.sh that requires Unix (including OS X and Linux) to run. But it should be easy to translate that script to a Windows batch file. Download the latest jsdoc_toolkit. Unpack the archive into, say, $HOME/jsdoc-toolkit. Make the script $HOME/jsdoc-toolkit/jsrun.sh executable and tell it where to look for the JSDoc binary and the template (which controls what the result looks like). JSDOCDIR="$HOME/local/jsdoc-toolkit" JSDOCTEMPLATEDIR="$JSDOCDIR/templates/jsdoc" Now you can move the script anywhere you want to, e.g. a bin/ directory. For the purpose of this demonstration, we don’t move the script. Use jsrun.sh on a directory of JavaScript files: $HOME/jsdoc-toolkit/jsrun.sh -d=$HOME/doc $HOME/js Input: $HOME/js – a directory of JavaScript files (see below for an example). Output: $HOME/doc – where to write the generated files. If you put the JavaScript code at the beginning of this post into a file $HOME/js/util.js then JSDoc produces the following files: $HOME/doc +-- files.html +-- index.html +-- symbols +-- _global_.html +-- src ¦ +-- util.js.html +-- util.html 2. Introduction: What is JSDoc? It’s a common programming problem: You have written JavaScript code that is to be used by others and need a nice-looking HTML documentation of its API. Java has pioneered this domain via its JavaDoc tool. The quasi-standard in the JavaScript world is JSDoc. As seen above, you document an entity by putting before it a special comment that starts with two asterisks. Templates. In order to output anything, JSDoc always needs a template, a mix of JavaScript and specially marked-up HTML that tells it how to translate the parsed documentation to HTML. JSDoc comes with a built-in template, but there are others that you can download [3]. 2.1. Terminology and conventions of JSDoc Doclet: JSDoc calls its comments doclets which clashes with JavaDoc terminology where such comments are called doc comments and a doclet is similar to a JSDoc template, but written in Java. Variable: The term variable in JSDoc often refers to all documentable entities which include global variables, object properties, and inner members. Instance properties: In JavaScript one typically puts methods into a prototype to share them with all instances of a class, while fields (non-function-valued properties) are put into each instance. JSDoc conflates shared properties and per-instance properties and calls them instance properties. Class properties, static properties: are properties of classes, usually of constructor functions. For example, Object.create is a class property of Object. Inner members: An inner member is data nested inside a function. Most relevant for documentation is instance-private data nested inside a constructor function. function MyClass() { var privateCounter = 0; // an inner member this.inc = function() { // an instance property privateCounter++; }; } 2.2. Syntax Let’s review the comment shown at the beginning: /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ This demonstrates some of the JSDoc syntax which consists of the following pieces. JSDoc comment: is a JavaScript block comment whose first character is an asterisk. This creates the illusion that the token /** starts such a comment. Tags: Comments are structured by starting lines with tags, keywords that are prefixed with an @ symbol. @param is an example above. HTML: You can freely use HTML in JSDoc comments; for example, to display a word in a monospaced font. Type annotations: You can document the type of a value by putting the type name in braces after the appropriate tags. Variations: Single type: @param {string} name Multiple types: @param {string|number} idCode Arrays of a type: @param {string[]} names Name paths: are used to refer to variables inside JSDoc comments. The syntax of such paths is as follows. myFunction MyConstructor MyConstructor.classProperty MyConstructor#instanceProperty MyConstructor-innerMember 2.3. A word on types There are two kinds of values in JavaScript: primitives and objects [7]. Primitive types: boolean, number, string. The values undefined and null are also considered primitive. Object types: All other types are object types, including arrays and functions. Watch out: the names of primitive types start with a lowercase letter. Each primitive type has a corresponding wrapper type, an object type with a capital name whose instances are objects: The wrapper type of boolean is Boolean. The wrapper type of number is Number. The wrapper type of string is String. Getting the name of the type of a value: Primitive value p: via typeof p. > typeof "" 'string' Compare: an instance of the wrapper type is an object. > typeof new String() 'object' Object value o: via o.constructor.name. Example: > new String().constructor.name 'String' 3. Basic tags Meta-data: @fileOverview: marks a JSDoc comment that describes the whole file. @author: Who has written the variable being documented? @deprecated: indicates that the variable is not supported, any more. It is a good practice to document what to use instead. @example: contains a code example, illustrating how the given entity should be used. /** * @example * var str = "abc"; * console.log(repeat(str, 3)); // abcabcabc */ Linking: @see: points to a related resource. /** * @see MyClass#myInstanceMethod * @see The Example Project. */ {@link ...}: works like @see, but can be used inside other tags. @requires resourceDescription: a resource that the documented entity needs. The resource description is either a name path or a natural language description. Versioning: @version versionNumber: indicates the version of the documented entity. Example: @version 10.3.1 @since versionNumber: indicates since which version the documented entity has been available. Example: @since 10.2.0 4. Documenting functions and methods For functions and methods, one can document parameters, return values, and exceptions they might throw. @param {paramType} paramName description: describes the parameter whose name is paramName. Type and description are optional. Examples: @param str @param str The string to repeat. @param {string} str @param {string} str The string to repeat. Advanced features: Optional parameter: @param {number} [times] The number of times is optional. Optional parameter with default value: @param {number} [times=1] The number of times is optional. @returns {returnType} description: describes the return value of the function or method. Either type or description can be omitted. @throws {exceptionType} description: describes an exception that might be thrown during the execution of the function or method. Either type or description can be omitted. 4.1. Inline type information (“inline doc comments”) There are two ways of providing type information for parameters and return values. First, you can add a type annotation to @param and @returns. /** * @param {String} name * @returns {Object} */ function getPerson(name) { } Second, you can inline the type information: function getPerson(/**String*/ name) /**Object*/ { } 5. Documenting variables and fields Fields are properties with non-function values. Because instance fields are often created inside a constructor, you have to document them there. @type {typeName}: What type does the documented variable have? Example: /** @constructor */ function Car(make, owner) { /** @type {string} */ this.make = make; /** @type {Person} */ this.owner = owner; } @type {Person} This tag can also be used to document the return type of functions, but @returns is preferable in this case. @constant: A flag that indicates that the documented variable has a constant value. @default defaultValue: What is the default value of a variable? Example: /** @constructor */ function Page(title) { /** * @default "Untitled" */ this.title = title || "Untitled"; } @property {propType} propName description: Document an instance property in the class comment. Example: /** * @class * @property {string} name The name of the person. */ function Person(name) { this.name = name; } Without this tag, instance properties are documented as follows. /** * @class */ function Person(name) { /** * The name of the person. * @type {string} */ this.name = name; } Which one of those styles to use is a matter of taste. @property does introduce redundancies, though. 6. Documenting classes JavaScript’s built-in means for defining classes are weak, which is why there are many APIs that help with this task [5]. These APIs differ, often radically, so you have to help JSDoc with figuring out what is going on. There are three basic ways of defining a class: Constructor function: You must mark a constructor function, otherwise it will not be documented as a class. That is, capitalization alone does not mark a function as a constructor. /** * @constructor */ function Person(name) { } @class is a synonym for @constructor, but it also allows you to describe the class – as opposed to the function setting up an instance (see tag documentation below for an example). API call and object literal: You need two markers. First, you need to tell JSDoc that a given variable holds a class. Second, you need to mark an object literal as defining a class. The latter is done via the @lends tag. /** @class */ var Person = makeClass( /** @lends Person# */ { say: function(message) { return "This person says: " + message; } } ); API call and object literal with a constructor method: If one of the methods in an object literal performs the task of a constructor (setting up instance data, [8]), you need to mark it as such so that fields are found by JSDoc. Then the documentation of the class moves to that method. var Person = makeClass( /** @lends Person# */ { /** * A class for managing persons. * @constructs */ initialize: function(name) { this.name = name; }, say: function(message) { return this.name + " says: " + message; } } ); Tags: @constructor: marks a function as a constructor. @class: marks a variable as a class or a function as a constructor. Can be used in a constructor comment to separate the description of the constructor (first line below) from the description of the class (second line below). /** * Creates a new instance of class Person. * @class Represents a person. */ Person = function() { } @constructs: marks a method in an object literal as taking up the duties of a constructor. That is, setting up instance data. In such a case, the class must be documented there. Works in tandem with @lends. @lends namePath: specifies to which class the following object literal contributes. There are two ways of contributing. @lends Person# – the object literal contributes instance properties to Person. @lends Person – the object literal contributes class properties to Person. 6.1. Inheritance, namespacing JavaScript has no simple support for subclassing and no real namespaces [6]. You thus have to help JSDoc see what is going on when you are using work-arounds. @extends namePath: indicates that the documented class is the subclass of another one. Example: /** * @constructor * @extends Person */ function Programmer(name) { Person.call(this, name); ... } // Remaining code for subclassing omitted @augments: a synonym for @extends. @namespace: One can use objects to simulate namespaces in JavaScript. This tag marks such objects. Example: /** @namespace */ var util = { ... }; 7. Meta-tags Meta-tags are tags that are added to several variables. You put them in a comment that starts with “/**#@+”. They are then added to all variables until JSDoc encounters the closing comment “/**#@-*/”. Example: /**#@+ * @private * @memberOf Foo */ function baz() {} function zop() {} function pez() {} /**#@-*/ 8. Rarely used tags @ignore: ignore a variable. Note that variables without /** comments are ignored, anyway. @borrows otherNamePath as this.propName: A variable is just a reference to somewhere else; it is documented there. Example: /** * @constructor * @borrows Remote#transfer as this.send */ function SpecialWriter() { this.send = Remote.prototype.transfer; } @description text: Provide a description, the same as all of the text before the first tag. Manual categorization. Sometimes JSDoc misinterprets what a variable is. Then you can help it via one of the following tags: Tag Mark variable as @function function @field non-function value @public public (especially inner variables) @private private @inner inner and thus also private @static accessible without instantiation Name and membership: @name namePath: override the parsed name and use the given name, instead. @memberOf parentNamePath: the documented variable is a member of the specified object. Not explained here: @event, see [2]. 9. Related reading jsdoc-toolkit - A documentation generator for JavaScript: JSDoc homepage on Google Code. Includes a link to the downloads. JSDoc wiki: the official documentation of JSDoc and source of this post. JSDoc wiki – TemplateGallery: lists available JSDoc templates. JSDoc wiki – TagReference: a handy cheat-sheet for JSDoc tags. Lightweight JavaScript inheritance APIs Modules and namespaces in JavaScript JavaScript values: not everything is an object Prototypes as classes – an introduction to JavaScript inheritance From http://www.2ality.com/2011/08/jsdoc-intro.html

One of the rules of simple design is the necessity to minimize the number of moving parts, like classes and methods, as long as the tests are satisfied and we are not accepting duplication or feeling the lack of an explicit concept. Thus, a rule that aids simple design is to use primitive types unless a field has already some behavior attached: we don't create a class for the user's name or the user's password; we just use some strings. As we make progress, however, we must be able to revise our decisions via refactoring: if a field gains some logic, this behavior shouldn't be modelled by methods in the containing class, but by a new object. The code in this new class can be reused, while the containing object will change from case to case and you will end up duplicating the same methods. Transforming a scalar value into an object is the essence of the Replace Data Value with Object refactoring. In most of the cases, a Value Object or a Parameter Object come out as a result: while DDD pursue Value Objects as concepts in the domain layer, this refactoring is more general and can be applied anywhere. For instance, in a project we started introducing Data Transfer Objects to model the data sent by the controller to a Service Layer. Data values in PHP In PHP, all scalar values are by nature data values as they cannot host methods: string, integers, and booleans are proper scalar. arrays are not scalar in the Perl or mathematical sense, but they are still a primitive type. On the borderline, we find some simple objects used as data containers in PHP: ArrayObjects. SplHeap and other SPL data structures. The classes on the borderline may host methods, but the original class is out of reach for modification, and an indirection has to be introduced."Local Extension" Steps Create the new class: it should contain as a private field just the value you want to substitute. The methods you immediately need have to be chosen between a constructor, getters, and setters (where needed). Change the field in the containing class. Update the constructor to also create the new object and populate the field, or accept injection (a rarer case). Update the original getter to delegate to the new one. Update the original setter to delegate to the new one (where present) or to create a new object. Run tests at the functional level; the changes should be propagated to the construction phases, while the external usage should not change very much. Example In the initial state, magic arrays are passed around. It's very easy to build an array where a key is missing or is called incorrectly. newPassword(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' )); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword($changePasswordData) { /* it's not interesting to do something here */ } } After the introduction of an ArrayObject extension, a little type safety is ensure and we gained a place to put methods at a little cost. newPassword(new ChangePasswordCommand(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' ))); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword(ChangePasswordCommand $changePasswordData) { /* it's not interesting to do something here */ } } class ChangePasswordCommand extends ArrayObject { } We add methods to implement logic on this object; in this case, validation logic; in general cases, any kind of code that should not be duplicated by the different clients. For a stricter implementation, wrap an array or another data structure (scalars, SPL objects) instead of extending ArrayObject as you gain immutability and encapsulation (but this kind of objects need little encapsulation.) class ChangePasswordCommand extends ArrayObject { public function __construct($data) { if (!isset($data['userId'])) { throw new Exception('User id is missing.'); } parent::__construct($data); } public function getPassword() { if ($this['newPassword'] != $this['repeatNewPassword']) { throw new Exception('Password do not match.'); } return $this['newPassword']; } } Being this a refactoring however, this is the less invasive kind of introduction of objects you can make as the client code can still use the ArrayAccess interface and treat the object as a scalar array.

About one year ago I wrote a blog post about JavaScriptSerializer and the Serialize and Deserialize methods it supports. Note: This blog post has been in draft for sometime now, so I decided to complete it and publish it. There might be situation when you want to serialize to JSON string only specific properties of a given class. You can do that using JavaScriptSerializer in combination with LINQ. Let’s say we have the following class definition public class Customer { public string Name { get; set; } public string Surname { get; set; } public string Email { get; set; } public int Age { get; set; } public bool Drinker { get; set; } public bool Smoker { get; set; } public bool Single { get; set; } } Next, lets create method that will create sample data for our demo private List GetListOfCustomers() { List customers = new List(); customers.Add(new Customer() { Name = "Hajan", Surname = "Selmani", Age = 25, Drinker = false, Smoker = false, Single = false, Email = "hajan@hajan.com" }); customers.Add(new Customer() { Name = "John", Surname = "Doe", Age = 29, Drinker = false, Smoker = true, Single = false, Email = "john@doe.com" }); customers.Add(new Customer() { Name = "Mark", Surname = "Moris", Age = 34, Drinker = true, Smoker = true, Single = true, Email = "mark@moris.com" }); return customers; } So, we have three customers with some property values for each of them. Now, lets serialize some of their properties using JavaScriptSerializer. First, you must put the following directive: using System.Web.Script.Serialization; Next, we create list of customers that will get the returned value from GetListOfCustomers method and we create instance of JavaScriptSerializer class List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); Now, lets say we want to serialize as JSON string and retrieve only the Age property data… We do that with only one simple line of code: //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age)); The result will be: Nice! Now, what if we want to serialize multiple properties at once, but not all class properties? string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age })); The result will be: You see, the result is an array of objects with the four properties and their corresponding values we have selected using the LINQ query above. You can see that integer and boolean values are without quotes, which is correct way of serialization. Now, you probably saw a difference somewhere? Namely, in the first example where we have selected only one property, there are only the values of the property (no property name), while in the second example we have the property name and it’s corresponding value… Why is it like that? It’s because in the second query, we use new { … } to specify multiple properties in the select statement. Therefore, the anonymous new { … } creates an object of each found item. So, if you are interested to make some more tests, run the following two lines of code: var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); and you will obviously see the difference. If we use the new { } way for single property selection, like in the following example string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); the result will be: The complete demo code used for this blog post: List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age )); string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age, x.Drinker })); var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); You can download the demo project here.

There are a number of problems with having a large number of records in memory. One way around this is to use direct memory, but this is too low level for most developers. Is there a way to make this more friendly? Limitations of large numbers of objects The overhead per object is between 12 and 16 bytes for 64-bit JVMs. If the object is relatively small, this is significant and could be more than the data itself. The GC pause time increases with the number of objects. Pause times can be around one second per GB of objects. Collections and arrays only support two billion elements Huge collections One way to store more data and still follow object orientated principles is have wrappers for direct ByteBuffers. This can be tedious to write, but very efficient. What would be ideal is to have these wrappers generated automatically. Small JavaBean Example This is an example of JavaBean which would have far more overhead than actual data contained. interface MutableByte { public void setByte(byte b); public byte getByte(); } It is also small enough that I can create billions of these on my machine. This example creates a List with 16 billion elements. final long length = 16_000_000_000L; HugeArrayList hugeList = new HugeArrayBuilder() {{ allocationSize = 4 * 1024 * 1024; capacity = length; }.create(); List list = hugeList; assertEquals(0, list.size()); hugeList.setSize(length); // add a GC to see what the GC times are like. System.gc(); assertEquals(Integer.MAX_VALUE, list.size()); assertEquals(length, hugeList.longSize()); byte b = 0; for (MutableByte mb : list) mb.setByte(b++); b = 0; for (MutableByte mb : list) { byte b2 = mb.getByte(); byte expected = b++; if (b2 != expected) assertEquals(expected, b2); } From start to finish, the heap memory used is as follows. with -verbosegc 0 sec - 3100 KB used [GC 9671K->1520K(370496K), 0.0020330 secs] [Full GC 1520K->1407K(370496K), 0.0063500 secs] 10 sec - 3885 KB used 20 sec - 4428 KB used 30 sec - 4428 KB used ... deleted ... 1380 sec - 4475 KB used 1390 sec - 4476 KB used 1400 sec - 4476 KB used 1410 sec - 4476 KB used The only GC is one triggered explicitly. Without the System.gc(); no GC logs appear. After 20 sec, the increase in memory used is from logging how much memory was used. Conclusion The library is relatively slow. Each get or set takes about 40 ns which really adds up when there are so many calls to make. I plan to work on it so it is much faster. ;) On the upside, it wouldn't be possible to create 16 billion objects with the memory I have, nor could it be put in an ArrayList, so having it a little slow is still better than not working at all. From http://vanillajava.blogspot.com/2011/08/collection-with-billions-of-entries.html

i often get asked about ‘rest to soap’ transformation use cases these days. using an soa gateway like securespan to perform this type of transformation at runtime is trivial to setup. with securespan in front of any existing web service (in the dmz for example), you can virtualize a rest version of this same service. using an example, here is a description of the steps to perform this conversion. imagine the geoloc web service for recording geographical locations. it has two methods, one for setting a location and one for getting a location. see below what this would look like in soap. request: 34802398402 response: 52.37706 4.889721 request: 34802398402 52.37706 4.889721 response: ok here is the equivalent rest target that i want to support at the edge. payloads could be xml, but let’s use json to make it more interesting. get /position/34802398402 http 200 ok content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } post /position/34802398402 content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } http 200 ok ok now let’s implement this rest version of the service using securespan. i’m assuming that you already have a securespan gateway deployed between the potential rest requesters and the existing soap web service. first, i will create a new service endpoint on the gateway for this service and assign anything that comes at the uri pattern /position/* to this service. i will also allow the http verbs get and post for this service. rest geoloc service properties next, let’s isolate the resource id from the uri and save this as a context variable named ‘trackerid’. we can use a simple regex assertion to accomplish this. also, i will branch on the incoming http verb using an or statement. i am just focusing on get and post for this example but you could add additional logic for other http verbs that you want to support for this rest service. regex for rest service resource identification policy branching for get vs post for get requests, the transformation is very simple, we just declare a message variable using a soap skeleton into which we refer to the trackerid variable. soap request template this soap message is routed to the existing web service and the essential elements are isolated using xpath assertions. processing soap response the rest response is then constructed back using a template response. template json response a similar logic is performed for the post message. see below for the full policy logic. complete policy you’re done for virtualizing the rest service. setting this up with securespan took less than an hour, did not require any change on the existing soap web service and did not require the deployment of an additional component. from there, you would probably enrich the policy to perform some json schema validation , some url and query parameter validation, perhaps some authentication, authorization , etc.

A new method is needed, as we are factoring out some lines of code. It would be nice to have it available on a class which already has all the fields to execute it, but unfortunately we cannot modify original code (because it's part of a library, or it's bundled with PHP.) A classic example of Introduce Foreign Method is cause by a missing method on ArrayObject, or SplQueue, or other external classes. The first place to place a new method is the object the code works with, since it's cohesive with the rest of the data. The second best place is the client object, although the method could get duplicated in different clients. A Foreign Method is a method created in the client code to complete the functionality of a source object. How it works The source object becomes a first additional argument of the Foreign Method: Python and some other languages reflects this with the self first argument, which makes refactoring Foreign Methods into normal ones easier. For a PHP- example, consider PHPUnit: I use Foreign Methods in my test cases to wrap $this->getMock() multiple usages. I factor away the adaptation to the Api (use of false arguments or MockBuilder) and leave as arguments the variability of the mocks, like the number of calls or the expected parameters. A Foreign Method is a work-around: if you have the possibility to change the source class, definitely go for that. Another alternative is to introduce a wrapper, when you can modify the lifecycle in order to inject a wrapper in the client code instead of the original object. It's not the case for PHPUnit (you extend the class), but it is for SplQueue and other collection objects; wrapping has also other benefits which will be shown in the next articles. Steps Create the method in the client class. Make the server object the first parameter of the method, if it cannot be passed already through $this (in case it's referenced by a field). Substitute the duplicated code with method calls. Fowler suggests also to comment the method calling it Foreign Method and saying it should be moved onto the server object when it will become possible. Example Initially, we're using an ArrayObject and continuing to ordering it after each addition of an element: addUrl('twitter.com'); $links->add('plus.google.com', 'Google+'); $links->add('facebook.com', 'Facebook'); $expected = "Facebook\n" . "Google+\n" . "twitter.com"; $this->assertEquals($expected, $links->__toString()); } } class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->links[$url] = $text; $this->links->asort(); } public function addUrl($url) { $this->links[$url] = $url; $this->links->asort(); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } } We introduce a Foreign Method, newLink(): class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->newLink($url, $text); } public function addUrl($url) { $this->newLink($url, $url); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); } } We also add some comments noting that if more and more Foreign Methods pop out, a radical solution would be needed: /** * Foreign Method of the ArrayObject. Should be moved onto a newly extracted * collaborator which wraps the ArrayObject, or an heap-like data structure * should be used. */ private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); }

Below is a list of some tools that can help you examine your Java source code for potential problems: 1. PMD from http://pmd.sourceforge.net/ License: PMD is licensed under a “BSD-style” license PMD scans Java source code and looks for potential problems like: * Possible bugs – empty try/catch/finally/switch statements * Dead code – unused local variables, parameters and private methods * Suboptimal code – wasteful String/StringBuffer usage * Overcomplicated expressions – unnecessary if statements, for loops that could be while loops * Duplicate code – copied/pasted code means copied/pasted bugs You can download everything from here, and you can get an overview of all the rules at the rulesets index page. PMD is integrated with JDeveloper, Eclipse, JEdit, JBuilder, BlueJ, CodeGuide, NetBeans/Sun Java Studio Enterprise/Creator, IntelliJ IDEA, TextPad, Maven, Ant, Gel, JCreator, and Emacs. 2. FindBug from http://findbugs.sourceforge.net License: L-GPL FindBugs, a program which uses static analysis to look for bugs in Java code. And since this is a project from my alumni university (IEEE – University of Maryland, College Park – Bill Pugh) , I have to definitely add this contribution to this list. 3. Clover from http://www.cenqua.com/clover/ License: Free for Open Source (more like a GPL) Measures statement, method, and branch coverage and has XML, HTML, and GUI reporting. and comprehensive plug-ins for major IDEs. * Improve Test Quality * Increase Testing Productivity * Keep Team on Track Fully integrated plugins for NetBeans, Eclipse , IntelliJ IDEA, JBuilder and JDeveloper. These plugins allow you to measure and inspect coverage results without leaving the IDE. Seamless Integration with projects using Apache Ant and Maven. * Easy integration into legacy build systems with command line interface and API. Fast, accurate, configurable, detailed coverage reporting of Method, Statement, and Branch coverage. Rich reporting in HTML, PDF, XML or a Swing GUI Precise control over the coverage gathering with source-level filtering. Historical charting of code coverage and other metrics. Fully compatible with JUnit 3.x & 4.x, TestNG, JTiger and other testing frameworks. Can also be used with manual, functional or integration testing. 4. Macker from http://innig.net/macker/ License: GPL Macker is a build-time architectural rule checking utility for Java developers. It’s meant to model the architectural ideals programmers always dream up for their projects, and then break — it helps keep code clean and consistent. You can tailor a rules file to suit a specific project’s structure, or write some general “good practice” rules for your code. Macker doesn’t try to shove anybody else’s rules down your throat; it’s flexible, and writing a rules file is part of the development process for each unique project. 5 EMMA from http://emma.sourceforge.net/ License: EMMA is distributed under the terms of Common Public License v1.0 and is thus free for both open-source and commercial development. Reports on class, method, basic block, and line coverage (text, HTML, and XML). EMMA can instrument classes for coverage either offline (before they are loaded) or on the fly (using an instrumenting application classloader). Supported coverage types: class, method, line, basic block. EMMA can detect when a single source code line is covered only partially. Coverage stats are aggregated at method, class, package, and “all classes” levels. Output report types: plain text, HTML, XML. All report types support drill-down, to a user-controlled detail depth. The HTML report supports source code linking. Output reports can highlight items with coverage levels below user-provided thresholds. Coverage data obtained in different instrumentation or test runs can be merged together. EMMA does not require access to the source code and degrades gracefully with decreasing amount of debug information available in the input classes. EMMA can instrument individial .class files or entire .jars (in place, if desired). Efficient coverage subset filtering is possible, too. Makefile and ANT build integration are supported on equal footing. EMMA is quite fast: the runtime overhead of added instrumentation is small (5-20%) and the bytecode instrumentor itself is very fast (mostly limited by file I/O speed). Memory overhead is a few hundred bytes per Java class. EMMA is 100% pure Java, has no external library dependencies, and works in any Java 2 JVM (even 1.2.x). 6. XRadar from http://xradar.sourceforge.net/ License: BSD (me thinks) The XRadar is an open extensible code report tool currently supporting all Java based systems. The batch-processing framework produces HTML/SVG reports of the systems current state and the development over time – all presented in sexy tables and graphs. The XRadar gives measurements on standard software metrics such as package metrics and dependencies, code size and complexity, code duplications, coding violations and code-style violations. 7. Hammurapi from Hammurapi Group License: (if anyone knows the license for this email me Venkatt.Guhesan at Y! dot com) Hammurapi is a tool for execution of automated inspection of Java program code. Following the example of 282 rules of Hammurabi’s code, we are offered over 120 Java classes, the so-called inspectors, which can, at three levels (source code, packages, repository of Java files), state whether the analysed source code contains violations of commonly accepted standards of coding. Relevant Links: http://en.sdjournal.org/products/articleInfo/93 http://wiki.hammurapi.biz/index.php?title=Hammurapi_4_Quick_Start 8. Relief from http://www.workingfrog.org/ License: GPL Relief is a design tool providing a new look on Java projects. Relying on our ability to deal with real objects by examining their shape, size or relative place in space it gives a “physical” view on java packages, types and fields and their relationships, making them easier to handle. 9. Hudson from http://hudson-ci.org/ License: MIT Hudson is a continuous integration (CI) tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. 10. Cobertura from http://cobertura.sourceforge.net/ License: GNU GPL Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage. 11. SonarSource from http://www.sonarsource.org/ (recommended by Vishwanath Krishnamurthi – thanks) License: LGPL Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture & Design, Duplications, Unit Tests, Complexity, Potential bugs, Coding rules, Comments. From http://mythinkpond.wordpress.com/2011/07/14/java-tools-for-source-code-optimization-and-analysis/

I have seen a number of tests where a JVM has 10K threads. However, what happens if you go beyond this? My recommendation is to consider having more servers once your total reaches 10K. You can get a decent server for $2K and a powerful one for $10K. Creating threads gets slower The time it takes to create a thread increases as you create more thread. For the 32-bit JVM, the stack size appears to limit the number of threads you can create. This may be due to the limited address space. In any case, the memory used by each thread's stack add up. If you have a stack of 128KB and you have 20K threads it will use 2.5 GB of virtual memory. Bitness Stack Size Max threads 32-bit 64K 32,073 32-bit 128K 20,549 32-bit 256K 11,216 64-bit 64K stack too small 64-bit 128K 32,072 64-bit 512K 32,072 Note: in the last case, the thread stacks total 16 GB of virtual memory. Java 6 update 26 32-bit,-XX:ThreadStackSize=64 4,000 threads: Time to create 4,000 threads was 0.522 seconds 8,000 threads: Time to create 4,000 threads was 1.281 seconds 12,000 threads: Time to create 4,000 threads was 1.874 seconds 16,000 threads: Time to create 4,000 threads was 2.725 seconds 20,000 threads: Time to create 4,000 threads was 3.333 seconds 24,000 threads: Time to create 4,000 threads was 4.151 seconds 28,000 threads: Time to create 4,000 threads was 5.293 seconds 32,000 threads: Time to create 4,000 threads was 6.636 seconds After creating 32,073 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.525 seconds 8,000 threads: Time to create 4,000 threads was 1.239 seconds 12,000 threads: Time to create 4,000 threads was 1.902 seconds 16,000 threads: Time to create 4,000 threads was 2.529 seconds 20,000 threads: Time to create 4,000 threads was 3.165 seconds After creating 20,549 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.526 seconds 8,000 threads: Time to create 4,000 threads was 1.212 seconds After creating 11,216 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=512 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) The Code MaxThreadsMain.java From http://vanillajava.blogspot.com/2011/07/java-what-is-limit-to-number-of-threads.html

as i'm sure you know python doesn't have a concept of private members. one trick that is sometimes used is to hide an object inside a python closure, and provide a proxy object that only permits limited access to the original object. here's a simple example of a hide function that takes an object and returns a proxy. the proxy allows you to access any attribute of the original, but not to set or change any attributes. def hide(obj): class proxy(object): __slots__ = () def __getattr__(self, name): return getattr(obj, name) return proxy() here it is in action: >>> class foo(object): ... def __init__(self, a, b): ... self.a = a ... self.b = b ... >>> f = foo(1, 2) >>> p = hide(f) >>> p.a, p.b (1, 2) >>> p.a = 3 traceback (most recent call last): ... attributeerror: 'proxy' object has no attribute 'a' after the hide function has returned the proxy object the __getattr__ method is able to access the original object through the closure . this is stored on the __getattr__ method as the func_closure attribute (python 2) or the __closure__ attribute (python 3). this is a "cell object" and you can access the contents of the cell using the cell_contents attribute: >>> cell_obj = p.__getattr__.func_closure[0] >>> cell_obj.cell_contents <__main__.foo object at 0x...> this makes hide useless for actually preventing access to the original object. anyone who wants access to it can just fish it out of the cell_contents . what we can't do from pure-python is*set* the contents of the cell, but nothing is really private in python - or at least not in cpython. there are two python c api functions, pycell_get and pycell_set , that provide access to the contents of closures. from ctypes we can call these functions and both introspect and modify values inside the cell object: >>> import ctypes >>> ctypes.pythonapi.pycell_get.restype = ctypes.py_object >>> py_obj = ctypes.py_object(cell_obj) >>> f2 = ctypes.pythonapi.pycell_get(py_obj) >>> f2 is f true >>> new_py_obj = ctypes.py_object(foo(5, 6)) >>> ctypes.pythonapi.pycell_set(py_obj, new_py_obj) 0 >>> p.a, p.b (5, 6) as you can see, after the call to pycell_set the proxy object is using the new object we put in the closure instead of the original. using ctypes may seem like cheating, but it would only take a trivial amount of c code to do the same. two notes about this code. it isn't (of course) portable across different python implementations don't ever do this, it's for illustration purposes only! still, an interesting poke around the cpython internals with ctypes. interestingly i have heard of one potential use case for code like this. it is alleged that at some point armin ronacher was using a similar technique in jinja2 for improving tracebacks. (tracebacks from templating languages can be very tricky because the compiled python code usually bears a quite distant relationship to the original text based template.) just because armin does it doesn't mean you can though...

As you probably saw by now, JetBrains just announced that they are working on a brand new statically typed JVM language called Kotlin. I am planning to write a post to evaluate how Kotlin compares to the other existing languages, but first, I’d like to take a slightly different angle and try to answer a question I have already seen asked several times: what’s the point? We already have quite a few JVM languages, do we need any more? Here are a few reasons that come to mind. 1) Coolness New languages are exciting! They really are. I’m always looking forward to learning new languages. The more foreign they are, the more curious I am, but the languages I really look forward to discovering are the ones that are close to what I already know but not identical, just to find out what they did differently that I didn’t think of. Allow me to make a small digression in order to clarify my point. Some time ago, I started learning Japanese and it turned out to be the hardest and, more importantly, the most foreign natural language I ever studied. Everything is different from what I’m used to in Japanese. It’s not just that the grammar, the syntax and the alphabets are odd, it’s that they came up with things that I didn’t even think would make any sense. For example, in English (and many other languages), numbers are pretty straightforward and unique: one bag, two cars, three tickets, etc… Now, did it ever occur to you that a language could allow several words to mean “one”, and “two”, and “three”, etc…? And that these words are actually not arbitrary, their usage follows some very specific rules. What could these rules be? Well, in Japanese, what governs the word that you pick is… the shape of the object that you are counting. That’s right, you will use a different way to count if the object is long, flat, a liquid or a building. Mind-boggling, isn’t it? Here is another quick example: in Russian, each verb exists in two different forms, which I’ll call A and B to simplify. Russian doesn’t have a future tense, so when you want to speak at the present tense, you’ll conjugate verb A in the present, and when you want the future, you will use the B form… in the present tense. It’s not just that you need to learn two verbs per verb, you also need to know which one is which if you want to get your tenses right. Oh and these forms also have different meanings when you conjugate them in past tenses. End of digression. The reason why I am mentioning this is because this kind of construct bends your mind, and this goes for natural languages as much as programming languages. It’s tremendously exciting to read new syntaxes this way. For that reason alone, the arrival of new languages should be applauded and welcome. Kotlin comes with a few interesting syntactic innovations of its own, which I’ll try to cover in a separate post, but for now, I’d like to come back to my original point, which was to give you reasons why you should be excited about Kotlin, so let’s keep going down the list. 2) IDE support None of the existing JVM languages (Groovy, Scala, Fantom, Gosu, Ceylon) have really focused much on the tooling aspect. IDE plug-ins exist for each of them, all with varying quality, but they are all an afterthought, and they suffer from this oversight. The plug-ins are very slow to mature, they have to keep up with the internals of a compiler that’s always evolving and which, very often, doesn’t have much regards for the tools built on top of it. It’s a painful and frustrating process for tool creators and tool users alike. With Kotlin, we have good reasons to think that the IDE support will be top notch. JetBrains is basically announcing that they are building the compiler and the IDEA support in lockstep, which is a great way to guarantee that the language will work tremendously well inside IDEA, but also that other tools should integrate nicely with it as well (I’m rooting for a speedy Eclipse plug-in, obviously). 3) Reified generics This is a pretty big deal. Not so much for the functionality (I’ll get back to this in the next paragraph) but because this is probably the very first time that we see a JVM language with true support for reified generics. This innovation needs to be saluted. Correction from the comments: Gosu has reified generics Having said that, I don’t feel extremely excited by this feature because overall, I think that reified generics come at too high a price. I’ll try to dedicate a full blog post to this topic alone, because it deserves a more thorough treatment. 4) Commercial support JetBrains has a very clear financial interest in seeing Kotlin succeed. It’s not just that they are a commercial entity that can put money behind the development of the language, it’s also that the success of the language would most likely mean that they will sell more IDEA licenses, and this can also turn into an additional revenue stream derived from whatever other tools they might come up with that would be part of the Kotlin ecosystem. This kind of commercial support for a language was completely unheard of in the JVM world for fifteen years, and suddenly, we have two instances of it (Typesafe and now JetBrains). This is a good sign for the JVM community. 5) Still no Java successors Finally, the simple truth is that we still haven’t found any credible Java successor. Java still reigns supreme and is showing no sign of giving away any mindshare. Pulling numbers out of thin air, I would say that out of all the code currently running on the JVM today, maybe 94% of it is in Java, 3% is in Groovy and 1% is in Scala. This 94% figure needs to go down, but so far, no language has stepped up to whittle it down significantly. What will it take? Obviously, nothing that the current candidates are offering (closures, modularity, functional features, more concise syntax, etc…) has been enough to move that needle. Something is still missing. Could the missing piece be “stellar IDE support” or “reified generics”? We don’t know yet because no contender offers any of these features, but Kotlin will, so we will soon know. Either way, I am predicting that we will keep seeing new JVM languages pop up at a regular pace until one finally claims the prize. And this should be cause for rejoicing for everyone interested in the JVM ecosystem. So let’s cheer for Kotlin and wish JetBrains the best of luck with their endeavor. I can’t wait to see what will come out of this. Oh, and secretly, I am rooting for Eclipse to start working on their own JVM language too, obviously. From http://beust.com/weblog/2011/07/20/five-reasons-why-should-rejoice-about-kotlin/

Sometimes there is too much logic to deal with in a single class. You tried extracting methods, but they are so many that the design is still complex to understand. The next step in the refactoring quest is Extract Class, the creation of a new class whose objects will be referenced from the original class. Fields and methods may be moved in the new class, in order for the original to get smaller and more manageable. Why class inflation happens? This refactoring is always caused by classes growing in responsibilities. My personal hypothesis is that we as developers have a bias for the add field|method operations preferring it to the add class. Usually, creating a new class also mean adding an entire new file (hopefully) and more design considerations like its namespace and name. The mental cost for the developer is heavier, but the results are often better than for smaller extractions, as classes can be reused independently; extracted methods are instead clustered together. Moreover, our libraries (e.g. ORMs such as Doctrine 2) reinforce this bias by making significatively more difficult to extract a Value Object (should you serialize it? write a custom DBAL type?) or even another Entity (should I link to it with a @OneToOne? @OneToMany? Which cascade options will work? Which constraints the relationships has?) By the way, this solution is a manifestation of composition over inheritance in the refactoring realm (while there are also other options where the class is made smaller by introducing superclasses instead of an unrelated type.) What are the signs it's time to extract a class? You may encounter a subset of methods and fields that cluster together: for example, they are identified by a prefix; or they have have a temporal coupling which makes them change together faster or slower than the other fields. They may be of similar types (scalar or superclass). Another option to target higher cohesion is to simply see which fields are used together by each method in the class. Fowler's suggestion is to try removing each field (conceptually), and think about which other fields become useless. Repeat this and you'll find the subsets of fields to extract if they exist. Steps Divide responsibilities in source class and extracted class: fields and methods should be classified to one or the two targets. This is true for public and private members, but the latter may change scope to public to be visible from the source class. Create a new class, and check the names of the source and the new one. In the extracted class, you decide the name on the fly; in the source class, you have to change the old name if it's no longer applicable (and later, also the name of references in the system to its objects). It may be the case that the extracted class steals the name of the source one. Place a field in the source class referencing an object (or more) of the extracted class. The field may be initialized in the constructor, and also injected with a constructor parameter if the change is not too invasive. Move Field iteratively from the source to the extracted class. Move Method iteratively. If you bundle steps 4 and 5, you'll be faster, but the point is you should be able to go in smaller steps when it is necessary. Likewise, TDD is taught with baby steps because it gives you the ability to make them when required: everyone is capable of cutting a giant piece of code and fiddling with it for hours until it works again. But that involves a rewriting part, not only refactoring. Reduce the interfaces that each class exposes. Often the extracted one only needs public methods for what the original class uses, while the original class maintains its old protocol to avoiding ripple effects towards the rest of the object graph. In fact, it's not even said that you should expose the extracted class. In many cases, you don't have to; but you'll be able to use it independently by creating other objects, while there is often no need to expose this particular object, composed by the source class (and the Law of Demeter says you shouldn't.) Tests should be executable after each movement of fields or methods. Example In the example, we pass from an initial state where formatting and HTML logic is crammed into the same class: assertEquals('10,000.00', $moneyAmount->toHtml()); } } class MoneyAmount { /** * @param int $amount */ public function __construct($amount) { $this->amount = $amount; } public function toHtml() { $amount = $this->amount; $formatted = ''; while (strlen($amount) > 3) { $cut = strlen($amount) % 3; $cut = $cut == 0 ? 3 : $cut; $formatted .= substr($amount, 0, $cut) . ','; $amount = substr($amount, $cut); } $formatted .= $amount . '.00'; $html = "$formatted"; return $html; } } To two separated classes, one modelling the logical amount and its formatting, one taking care of printing HTML tags. assertEquals('10,000.00', $moneyAmount->toHtml()); } } class MoneySpan { /** * @param int $amount */ public function __construct(MoneyAmount $amount) { $this->amount = $amount; } public function toHtml() { $html = '' . $this->amount->format() . ''; return $html; } } class MoneyAmount { private $amount; public function __construct($amount) { $this->amount = $amount; } public function format() { $amount = $this->amount; $formatted = ''; while (strlen($amount) > 3) { $cut = strlen($amount) % 3; $cut = $cut == 0 ? 3 : $cut; $formatted .= substr($amount, 0, $cut) . ','; $amount = substr($amount, $cut); } return $formatted . $amount . '.00'; } } You can see the four intermediate steps in the Github history of the file.

Learn more about Java collection performance in this post.

More than a few times recently we’ve wanted to use pattern matching on a collection of pairs/tuples and have run into trouble doing so. It’s easy enough if you don’t try and pattern match: > List(("Mark", 4), ("Charles", 5)).filter(pair => pair._2 == 4) res6: List[(java.lang.String, Int)] = List((Mark,4)) But if we try to use pattern matching: List(("Mark", 4), ("Charles", 5)).filter(case(name, number) => number == 4) We end up with this error: :1: error: illegal start of simple expression List(("Mark", 4), ("Charles", 5)).filter(case(name, number) => number == 4) It turns out that we can only use this if we pass the function to filter using {} instead of (): > List(("Mark", 4), ("Charles", 5)).filter { case(name, number) => number == 4 } res7: List[(java.lang.String, Int)] = List((Mark,4)) It was pointed out to me on the Scala IRC channel that the reason for the compilation failure has nothing to do with trying to do a pattern match inside a higher order function but that it’s not actually possible to use a case token without the {}. [23:16] mneedham: hey – trying to understand how pattern matching works inside higher order functions. Don’t quite get this code -> https://gist.github.com/1079110 any ideas? [23:17] dwins: mneedham: scala requires that “case” statements be inside curly braces. nothing to do with higher-order functions [23:17] mneedham: is there anywhere that’s documented or is that just a known thing? [23:18] mneedham: I expected it to work in normal parentheses [23:21] amacleod: mneedham, it’s documented. Whether it’s documented simply as “case statements need to be in curly braces” is another question The first line of Section 8.5 ‘Pattern Matching Anonymous Functions’ of the Scala language spec proves what I was told: Syntax: BlockExpr ::= ‘{’ CaseClauses ‘} It then goes into further detail about how the anonymous function gets converted into a pattern matching statement which is quite interesting reading. From http://www.markhneedham.com/blog/2011/07/12/scala-pattern-matching-a-pair-inside-mapfilter/

Tomcat 7 has been improved a lot and along with everything else that it brings, a very nice feature is provided - an API for embedding Tomcat into your application. The API was provided in earlier versions of Tomcat but it was quite cumbersome to use. To to start the embedded version of Tomcat one may need to build the required JARs. svn co https://svn.apache.org/repos/asf/tomcat/trunk tomcat cd tomcat ant embed-jars ls -l output/embed total 5092 -rw-r--r-- 1 anton None 56802 2011-03-06 17:09 LICENSE -rw-r--r-- 1 anton None 1194 2011-03-06 17:09 NOTICE -rw-r--r-- 1 anton None 1690519 2011-03-06 17:09 ecj-3.6.jar -rw-r--r-- 1 anton None 234625 2011-03-06 17:09 tomcat-dbcp.jar -rw-r--r-- 1 anton None 2402517 2011-03-06 17:09 tomcat-embed-core.jar -rw-r--r-- 1 anton None 781989 2011-03-06 17:09 tomcat-embed-jasper.jar -rw-r--r-- 1 anton None 34106 2011-03-06 17:09 tomcat-embed-logging-juli.jar The following snippet demonstrates the embedded Tomcat usage with a deployed servlet instance. import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.File; import java.io.IOException; import java.io.Writer; public class Main { public static void main(String[] args) throws LifecycleException, InterruptedException, ServletException { Tomcat tomcat = new Tomcat(); tomcat.setPort(8080); Context ctx = tomcat.addContext("/", new File(".").getAbsolutePath()); Tomcat.addServlet(ctx, "hello", new HttpServlet() { protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { Writer w = resp.getWriter(); w.write("Hello, World!"); w.flush(); } }); ctx.addServletMapping("/*", "hello"); tomcat.start(); tomcat.getServer().await(); } } The only two JARs required are tomcat-embed-core.jar and tomcat-embed-logging-juli.jar. It means that there will be no JSP support and pooling will also be disabled. But that's enough to start a servlet and in most cases that is what you probably need. From http://arhipov.blogspot.com/2011/03/embedded-tomcat-minimal-version.html

In my daily work, I use both an RDBMS and MarkLogic, an XML database. MarkLogic can be considered akin to the newer NoSQL databases, but it has the added structure of XML and standard languages in XQuery and XPath. The NoSQL databases are typically storing documents or key-value pairs, and some other things in between. Given that any datastore will be searched at some point, you will always care how the data is actually stored or whether there is some way to query it easily. Once you start thinking about the problem, you quickly generalize to the “how do I persist any type of data” question. However, my focus is not going to be the comparison of the various data stores, but the comparison of how data is stored. More specifically, I want to show the object serialization, mainly the Java built in method, as a data persistence format is evil. Given what you normally read on this blog, this may seem like an oddly timed post, but I have run into serialization issues lately in some production code and Mark Needham recently wrote an interesting post about this as well. Coincidentally, Mark is also working with MarkLogic, and there is an interesting item in his post: The advantage of doing things this way [using lightweight wrappers] is that it means we have less code to write than we would with the serialisation/deserialisation approach although it does mean that we’re strongly coupled to the data format that our storage mechanism uses. However, since this is one bit of the architecture which is not going to change it seems to makes sense to accept the leakage of that layer. The interesting part of this is that he has accepted using the data format of the storage mechanism, XML in MarkLogic in this case. Why is this interesting? First, it is a move away from the ORM technologies that try to hide the complexities of converting data into objects in the RDBMS world. Also, this is a glimpse into the types of issues that could arise from non-RDBMS storage choices as well as how to persist objects in general. So, an RDBMS is typically used to map object attributes to a table and columns. The mapping is mostly straightforward with some defined relationship for child objects and collections. This is a well-known area, called Object-Relational Mapping (ORM), and several open source and commercial options exist. In this scenario, object attributes are stored in a similar datatype, meaning a String is stored as a varchar and an int is stored as an integer. But, what happens when you move away from an RDBMS for data persistence? If you look at Java and its session objects, pure object serialization is used. Assuming that an application session is fairly short-lived, meaning at most a few hours, object serialization is simple, well supported and built into the Java concept of a session. However, when the data persistence is over a longer period of time, possibly days or weeks, and you have to worry about new releases of the application, serialization quickly becomes evil. As any good Java developer knows, if you plan to serialize an object, even in a session, you need a real serialization ID (serialVersionUID), not just a 1L, and you need to implement the Serializable interface. However, most developers do not know the real rules behind the Java deserialization process. If your object has changed, more than just adding simple fields to the object, it is possible that Java cannot deserialize the object correctly even if the serialization ID has not changed. Suddenly, you cannot retrieve your data any longer, which is inherently bad. Now, may developers reading this may say that they would never write code that would have this problem. That may be true, but what about a library that you use or some other developer no longer employed by your company? Can you guarantee that this problem will never happen? The only way to guarantee that is to use a different serialization method. What options do we have? Obviously, there are the NoSQL datastores but the actual object format is the relevant question not which solution to choose. Besides the obvious serialized object, some NoSQL datastores use JSON to store objects, MarkLogic uses XML and there are others that store just key-value pairs. Key-value pairs are typically a mapping of a text key to a value that is a serialized object, either a binary or textual format. So, that leaves us with XML, JSON and other textual formats. One of the benefits of a structured format like XML or JSON is that they can be made searchable and provide some level of context. I have talked about data formats before, so I won’t go into a comparison again. However, do these types of formats avoid the issues that native Java object serialization has? This is really dependent upon what library you are using for serialization. Some libraries will deserialize an object without any issues regardless of whether the object field list has changed. Other libraries could have problems depending upon whether a serialized field exists in the target object, or there might not be solid support for collections (though that is doubtful at this point). Given that even structured formats could have serialization issues, is the only safe path hand-coded mappings like those used by ORM tools? Some JSON and XML serialization tools use the same mapping methods as the ORM tools in order to avoid these problems. However, once you define these mappings, you are explicitly stating how an object gets translated. This explicit definition will require maintenance, but that is definitely cleaner than trying to trace down a serialization defect in some random stack trace. So is implicit object serialization really worth the potential headaches? Or should we just consider it evil and never speak of it again? From http://regulargeek.com/2011/07/06/is-object-serialization-evil/

Today I'll talk about a famous problem : restarting a Java application. It is especially useful when changing the language of a GUI application, so that we need to restart it to reload the internationalized messages in the new language. Some look and feel also require to relaunch the application to be properly applied. A quick Google search give plenty answers using a simple : Runtime.getRuntime().exec("java -jar myApp.jar"); System.exit(0); This indeed basically works, but this answer that does not convince me for several reasons : 1) What about VM and program arguments ? (this one is secondary in fact, because can be solve it quite easily). 2) What if the main is not a jar (which is usually the case when launching from an IDE) ? 3) Most of all, what about the cleaning of the closing application ? For example if the application save some properties when closing, commit some stuffs etc. 4) We need to change the command line in the code source every time we change a parameter, the name of the jar, etc. Overall, something that works fine for some test, sandbox use, but not a generic and elegant way in my humble opinion. Ok, so my purpose here is to implement a method : public static void restartApplication(Runnable runBeforeRestart) throws IOException { ... } that could be wrapped in some jar library, and could be called, without any code modification, by any Java program, and by solving the 4 points raised previously. Let's start by looking at each point and find a way to answer them in an elegant way (let's say the most elegant way that I found). 1) How to get the program and VM arguments ? Pretty simple, by calling a : ManagementFactory.getRuntimeMXBean().getInputArguments(); Concerning the program arguments, the Java property sun.java.command we'll give us both the main class (or jar) and the program arguments, and both will be useful. String[] mainCommand = System.getProperty("sun.java.command").split(" "); 2) First retrieve the java bin executable given by the java.home property : String java = System.getProperty("java.home") + "/bin/java"; The simple case is when the application is launched from a jar. The jar name is given by a mainCommand[0], and it is in the current path, so we just have to append the application parameters mainCommand[1..n] with a -jar to get the command to execute : String cmd = java + vmArgsOneLine + "-jar " + new File(mainCommand[0]).getPath() + mainCommand[1..n]; We'll suppose here that the Manifest of the jar is well done, and we don't need to specify the main nor the classpath. Second case : when the application is launched from a class. In this case, we'll specify the class path and the main class : String cmd = java + vmArgsOneLine + -cp \"" + System.getProperty("java.class.path") + "\" " + mainCommand[0] + mainCommand[1..n]; 3) Third point, cleaning the old application before launching the new one. To do such a trick, we'll just execute the Runtime.getRuntime().exec(cmd) in a shutdown hook. This way, we'll be sure that everything will be properly clean up before creating the new application instance. Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { ... } }); Run the runBeforeRestart that contains some custom code that we want to be executed before restarting the application : if(runBeforeRestart != null) { runBeforeRestart.run(); } And finally, call the System.exit(0);. And we're done. Here is our generic method : /** * Sun property pointing the main class and its arguments. * Might not be defined on non Hotspot VM implementations. */ public static final String SUN_JAVA_COMMAND = "sun.java.command"; /** * Restart the current Java application * @param runBeforeRestart some custom code to be run before restarting * @throws IOException */ public static void restartApplication(Runnable runBeforeRestart) throws IOException { try { // java binary String java = System.getProperty("java.home") + "/bin/java"; // vm arguments List vmArguments = ManagementFactory.getRuntimeMXBean().getInputArguments(); StringBuffer vmArgsOneLine = new StringBuffer(); for (String arg : vmArguments) { // if it's the agent argument : we ignore it otherwise the // address of the old application and the new one will be in conflict if (!arg.contains("-agentlib")) { vmArgsOneLine.append(arg); vmArgsOneLine.append(" "); } } // init the command to execute, add the vm args final StringBuffer cmd = new StringBuffer("\"" + java + "\" " + vmArgsOneLine); // program main and program arguments String[] mainCommand = System.getProperty(SUN_JAVA_COMMAND).split(" "); // program main is a jar if (mainCommand[0].endsWith(".jar")) { // if it's a jar, add -jar mainJar cmd.append("-jar " + new File(mainCommand[0]).getPath()); } else { // else it's a .class, add the classpath and mainClass cmd.append("-cp \"" + System.getProperty("java.class.path") + "\" " + mainCommand[0]); } // finally add program arguments for (int i = 1; i < mainCommand.length; i++) { cmd.append(" "); cmd.append(mainCommand[i]); } // execute the command in a shutdown hook, to be sure that all the // resources have been disposed before restarting the application Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { try { Runtime.getRuntime().exec(cmd.toString()); } catch (IOException e) { e.printStackTrace(); } } }); // execute some custom code before restarting if (runBeforeRestart!= null) { runBeforeRestart.run(); } // exit System.exit(0); } catch (Exception e) { // something went wrong throw new IOException("Error while trying to restart the application", e); } } From : http://lewisleo.blogspot.jp/2012/08/programmatically-restart-java.html

One thing I’m doing very often and always searching on the Internet is how to obtain a self-signed SSL certificate and install it in both my client browsers and my local Tomcat. Sure enough there are enough resources available online, but since it’s a bore to go looking for the right one (yes, some do not work), I figured let’s do it right once and document it so that it will always be there. Create the keystore Keystores are, guess what, files where your store your keys. In our case, we need to create one that will be used by both Tomcat and for the certificat generation. The command-line is: keytool -genkey -keyalg RSA -alias blog.frankel.ch -keystore keystore.jks -validity 999 -keysize 2048 The parameters are as follow: Parameter Value Description -genkey Requests the keytool to generate a key. For all provided features, type keytool -help -keyalg RSA Wanted algorithm. The specified algorithm must be made available by one of the registered cryptographic service providers -keysize 2048 Key size -validity 999 Validity in days -alias blog.frankel.ch Entry in the keystore -keystore keystore.jks Keystore. If the keystore doesn’t exist yet, it will be created and you’ll be prompted for a new password; otherwise, you’ll prompted for the current store’s password Configure Tomcat Tomcat’s SSL configuration is done in the ${TOMCAT_HOME}/conf/server.xml file. Locate the following snippet: Now, uncomment it and add the following attributes: keystoreFile="/path/to/your/keystore.jks" keystorePass="Your password" Note: if the store only contains a single entry, fine; otherwise, you’ll need to configure the entry’s name with keyAlias="blog.frankel.ch" Starting Tomcat and browsing to https://localhost:8443/ will show you Tomcat’s friendly face. Additionnaly, the logs will display: 28 june 2011 20:25:14 org.apache.coyote.AbstractProtocolHandler init INFO: Initializing ProtocolHandler ["http-bio-8443"] Export the certificate The certificate is created from our previous entry in the keystore. The command-line is: keytool -export -alias blog.frankel.ch -file blog.frankel.ch.crt -keystore keystore.jks Even simpler, we are challenged for the keystore’s password and that’s all. The newly created certificate is now available in the filesystem. We just have to distribute it to all browsers that will connect to Tomcat in order to bypass security warnings (since it’s a self-signed certificate). Spread the word The last step is to put the self-signed certificate in the list of trusted certificates in Firefox. For a quick and dirty way, import it in your own Firefox (Options -> Advanced -> Show certificates -> Import…) and distribute the %USER_HOME%"/Application Data/Mozilla/Firefox/Profiles/xzy.default/cert8.db file. It has to be copied to the %FIREFOX_HOME%/defaults/profile folder so that every single profile on the target machine is updated. Note that this way of doing will lose previously individually accepted certificates (in short, we’re overwriting the whole certificate database). For a more industrial process, look at the next section. To go further: The Most Common Java Keytool Keystore Commands Tomcat 7 SSL Configuration HOW-TO Where can I download certutil.exe for Windows From http://blog.frankel.ch/ssl-your-tomcat-7

This tutorial will walk you through how to configure SSL (https://localhost:8443 access) on Tomcat in 5 minutes. For this tutorial you will need: Java SDK (used version 6 for this tutorial) Tomcat (used version 7 for this tutorial) The set up consists in 3 basic steps: Create a keystore file using Java Configure Tomcat to use the keystore Test it (Bonus ) Configure your app to work with SSL (access through https://localhost:8443/yourApp) 1 – Creating a Keystore file using Java Fisrt, open the terminal on your computer and type: Windows: cd %JAVA_HOME%/bin Linux or Mac OS: cd $JAVA_HOME/bin The $JAVA_HOME on Mac is located on “/System/Library/Frameworks/JavaVM.framework/Versions/{your java version}/Home/” You will change the current directory to the directory Java is installed on your computer. Inside the Java Home directory, cd to the bin folder. Inside the bin folder there is a file named keytool. This guy is responsible for generating the keystore file for us. Next, type on the terminal: keytool -genkey -alias tomcat -keyalg RSA When you type the command above, it will ask you some questions. First, it will ask you to create a password (My password is “password“): loiane:bin loiane$ keytool -genkey -alias tomcat -keyalg RSA Enter keystore password: password Re-enter new password: password What is your first and last name? [Unknown]: Loiane Groner What is the name of your organizational unit? [Unknown]: home What is the name of your organization? [Unknown]: home What is the name of your City or Locality? [Unknown]: Sao Paulo What is the name of your State or Province? [Unknown]: SP What is the two-letter country code for this unit? [Unknown]: BR Is CN=Loiane Groner, OU=home, O=home, L=Sao Paulo, ST=SP, C=BR correct? [no]: yes Enter key password for (RETURN if same as keystore password): password Re-enter new password: password It will create a .keystore file on your user home directory. On Windows, it will be on: C:Documents and Settings[username]; on Mac it will be on /Users/[username] and on Linux will be on /home/[username]. 2 – Configuring Tomcat for using the keystore file – SSL config Open your Tomcat installation directory and open the conf folder. Inside this folder, you will find the server.xml file. Open it. Find the following declaration: Uncomment it and modify it to look like the following: Connector SSLEnabled="true" acceptCount="100" clientAuth="false" disableUploadTimeout="true" enableLookups="false" maxThreads="25" port="8443" keystoreFile="/Users/loiane/.keystore" keystorePass="password" protocol="org.apache.coyote.http11.Http11NioProtocol" scheme="https" secure="true" sslProtocol="TLS" /> Note we add the keystoreFile, keystorePass and changed the protocol declarations. 3 – Let’s test it! Start tomcat service and try to access https://localhost:8443. You will see Tomcat’s local home page. Note if you try to access the default 8080 port it will be working too: http://localhost:8080 4 – BONUS - Configuring your app to work with SSL (access through https://localhost:8443/yourApp) To force your web application to work with SSL, you simply need to add the following code to your web.xml file (before web-app tag ends): securedapp /* CONFIDENTIAL The url pattern is set to /* so any page/resource from your application is secure (it can be only accessed with https). The transport-guarantee tag is set to CONFIDENTIAL to make sure your app will work on SSL. If you want to turn off the SSL, you don’t need to delete the code above from web.xml, simply change CONFIDENTIAL to NONE. Reference: http://tomcat.apache.org/tomcat-7.0-doc/ssl-howto.html (this tutorial is a little confusing, that is why I decided to write another one my own). Happy Coding! From http://loianegroner.com/2011/06/setting-up-ssl-on-tomcat-in-5-minutes-httpslocalhost8443/

After one month of evaluation of frameworks and tools, I choose ExtJS for UI and Spring/SpringMVC for the business layer for my pet project. By using ExtJS we can send data to server by form submits or as request parameters, or in JSON format through Ajax requests. ExtJS uses JSON format in many situations to hold data. So I thought using JSON as data exchange format between EXTJS and Spring would be consistent. The following code snippets explain how we can use ExtJS and SpringMVC3 to exchange data in JSON format. 1. Register MappingJacksonHttpMessageConverter in dispatcher-servlet.xml Add jackson-json jar(s) to WEB-INF/lib 2. Trigger the POST request from ExtJS script as follows: Ext.Ajax.request({ url : 'doSomething.htm', method: 'POST', headers: { 'Content-Type': 'application/json' }, params : { "test" : "testParam" }, jsonData: { "username" : "admin", "emailId" : "admin@sivalabs.com" }, success: function (response) { var jsonResp = Ext.util.JSON.decode(response.responseText); Ext.Msg.alert("Info","UserName from Server : "+jsonResp.username); }, failure: function (response) { var jsonResp = Ext.util.JSON.decode(response.responseText); Ext.Msg.alert("Error",jsonResp.error); } }); 3. Write a Spring Controller to handle the "/doSomething.htm" reguest. @Controller public class DataController { @RequestMapping(value = "/doSomething", method = RequestMethod.POST) @ResponseBody public User handle(@RequestBody User user) throws IOException { System.out.println("Username From Client : "+user.getUsername()); System.out.println("EmailId from Client : "+user.getEmailId()); user.setUsername("SivaPrasadReddy"); user.setEmailId("siva@sivalabs.com"); return user; } } Any other better approaches? From http://sivalabs.blogspot.com/2011/06/how-to-post-and-get-json-between-extjs.html