Alois Cochard's Blog

A simple (REST) web service client in Scala

2011-05-16T15:28:00.000+02:00

For the purpose of creating a nice API for my new project Caligo (more on this in a future blog post), I was looking for a simple solution to access REST web service in Scala.

My requirement was simple: access an HTTP web service, and exchange data with him using the JSON format.

During my experiment I came across theses two nice Scala libraries:

Dispatch: It's a Scala DSL build on the rock-solid Apache HttpClient
SJSON: A well designed JSON serializer with reflection support (created by Debasish Gosh)

Dispatch seems to be the only available HTTP client for Scala today, and since it's based on the Apache's one no need to worry about his reliability or compatibility.

SJSON was choose against Lift-JSON due to it's better handling of reflection on beans, both of them did a great job on case classes, but only SJSON was effective on beans (was mandatory for me since I must do polymorphism on my model and case classes do not support this).

Lift-JSON support of beans (thru reflection) must be improved on the version targeting Scala 2.9

Here is a sample usage of the frameworks, both of them are easily integrated, and using the power of the Scala syntax a full client request doesn't take that much code to be implemented (in this case 4 lines of code, model included):

Bring Your Code: An algorithm for index translation

2010-06-30T19:19:00.000+02:00

I'm actually working on a text-mining/semantic web application focused (for the moment) on biomedical informations and developed in Java.
We are using external tools for text-mining analysis and unfortunatly theses tools don't handle HTML pretty well ...
If we send raw HTML to the text-mining service, he simply break.
So we must convert HTML to plain-text before processing text, and because the tools return identified words by giving their positions, we must translate theses position (or indexes) to find corresponding word in the original HTML.

Bring Your Code
I've searched the web but didn't find any code that I can reuse, so I created a simple 'IndexTranslator' class.
I'm sure that my implementation of this algorithm isn't the best (a bit under pressure due to production deployment), so I'm looking for a better one.
Keep in mind that this code is used in an enterprise application and must be relatively easy to maintain (even it's a bit less efficient) .

I created a gist with the implementation and the corresponding JUnit test case to let you fork the code and see if it work as I expect.

IndexTranslator.java

IndexTranslatorTest.java

Let's fork !

By the way, if you are aware of a library implementing this, I'm ready to send this code to the garbage and use the library instead !

Vim is everywhere

2010-06-22T13:03:00.003+02:00

I'm now sure that vim is the perfect tool to write *clean* code !

And for vim fanatics, you can even get vim inside NetBeans (integration work much better than in Eclipse):
http://jvi.sourceforge.net/

Spring Batch integration module for GridGain

2010-04-30T23:15:00.005+02:00

For the purpose of using Spring Batch in a scalable and distributed manner to process huge amount of data, I am actually developing some components to make integration of Spring Batch with compute/data grid easier.

Different solutions is offered by Spring Batch to provide scalability, the one that best suit my needs is remote chunking.

As I already done some investigation before using GridGain I chose this framework to implement a distributed remote chunking system that can be easily integrated into any existing Spring Batch systems.

Using GridGain is really straightforward, and setting up a grid on a development machine doesn't need so much configuration.

The only issue I faced is due to the fact that GridGain use serialization to deploy tasks on nodes, in order to be able to deploy a remote ChunkProcessor, it must contains serializable ItemProcessor and ItemWriter, which unfortunately is not the case by default.

So instead of creating new interfaces, I made a SerializableChunkProcessor which only accept serializable ItemProcessor and ItemWriter. It's surely not the smarter solution, but since I can't modify default interfaces in Spring Batch and I don't want to create my own interfaces, this workaround will suffice.

Usage
Here is the job application context used for the integration test, as you can see the 'real' ItemProcessor / ItemWriter are injected into the GridGain chunk writer:

<bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="resource" value="classpath:sentences.txt"/> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="fieldSetMapper"> <bean class="org.springframework.batch.integration.chunk.gridgain.SentenceFieldSetMapper"/> </property> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="delimiter" value="\"/> </bean> </property> </bean> </property> </bean> <bean id="processor" class="org.springframework.batch.integration.chunk.gridgain.WordCounterProcessor"/> <bean id="writer" class="org.springframework.batch.integration.chunk.gridgain.ConsoleWriter"/>  <bean id="chunkWriter" class="org.springframework.batch.integration.chunk.gridgain.ChunkGridGainItemWriter"> <property name="chunkProcessor"> <bean class="org.springframework.batch.integration.chunk.SerializableChunkProcessor"> <property name="itemProcessor" ref="processor"/> <property name="itemWriter" ref="writer"/> </bean> </property> <property name="grid" ref="grid"/> <property name="throttleLimit" value="16"/> </bean>  <job id="gridgainJob" xmlns="http://www.springframework.org/schema/batch"> <step id="step1"> <tasklet> <chunk reader="reader" writer="chunkWriter" commit-interval="4"/> </tasklet> </step> </job>

Download
You can download the spring-batch-integration-gridgain module here:
http://github.com/downloads/aloiscochard/spring-batch-integration-gridgain/spring-batch-integration-gridgain-0.0.1-SNAPSHOT.jar

If you want to see a full working sample, take a look at the integration test. The full project sources can be downloaded here:
http://github.com/aloiscochard/spring-batch-integration-gridgain

Integrating failover strategy for Spring Security NTLM

2010-03-25T21:41:00.006+01:00

During implementation of the NTLM authentication into our application, I wanted to achieve failover to standard login page (html form) if NTLM authentication fail.

In order to obtain the wanted behavior I extended some Spring Security classes and hack them a little bit, so I created:

An AuthenticationEntryPoint strategy to switch between NTLM or standard login
A custom AuthenticationProvider to prevent password checking when user authenticate with NTLM
A custom NtlmProcessingFilter to disable NTLM authentication if a remember-me cookie is present

The thing I was unable to do is detecting if the user client is NTLM compliant before starting the NTLM challenge. Due to the way the protocol is build (and certainly for security reasons) it's impossible to know if the client is NTLM compliant before launching the challenge.

So let's took the scenario of a Google Chrome user:

A popup (HTTP realm) authentication window is shown to the user
He hit enter with invalid credentials (i.e. empty username/password)
The browser is redirected to the standard html login form

By using 'remember me' the NTLM authentication won't be shown next time the user access the site due to the presence of the cookie.

If the user client is Internet Explorer no authentication window is shown because the navigator use directly the information of the logged user. If the authentication fail, then the standard html login form is displayed letting him login with other credentials.

This implementation was made using Spring Security 2.0.5, but can be easly converted to Spring Security 3.0 using the Spring Security NTLM 3 module I recently created. Don't hesitate to ask me if you want that I publish an updated version.

Download
You can download a sample application including hacked class here:
http://github.com/downloads/aloiscochard/spring-security-ntlm-samples/spring-security-ntlm-samples-2.0.5.SNAPSHOT.zip

Spring Security NTLM 3

2010-03-21T22:25:00.002+01:00

As of version 3.X Spring Security doesn't include the NTLM extension anymore.

For the purpose of using NTLM authentication on Spring 3 projects, I migrated the 2.0.5 NTLM extension to the new version of Spring Security.

The full project sources are available here:
http://github.com/aloiscochard/spring-security-ntlm

I haven't took time fixing the following bugs found on version 2.0.5:
https://jira.springsource.org/browse/SEC-1003
https://jira.springsource.org/browse/SEC-1087

If your are interested to help correcting this defects don't hesitate to get in touch.

Download
You can download the spring-security-ntlm snapshot here:
http://github.com/downloads/aloiscochard/spring-security-ntlm/spring-security-ntlm-3.0.2.SNAPSHOT.jar

Usage
Due to the fact that NTLM isn't integrated in new version of spring-security-core, you need to add a custom filter.
Here is a snippet describing briefly how to do that:
Don't hesitate to ask if you need help, this extension work exactly as version 2.0.5. You can use an example from version 2.0.5 if you are starting from scratch.

Integrating Spring Security with NTLM + Crowd

2009-12-19T11:55:00.003+01:00

For the purpose of implementing a SSO (Single Sign On) system into our application, I investigated the use of spring-security and successfully implemented it using NTLM and Crowd.

Download Sample Project

Technologies
As most of our users are using Internet Explorer, NTLM seems to be the best solution. By the way, as developer we are using Firefox and fortunately Firefox can provide too NTLM credentials (see this to enableNTLM inside Firefox).
NTLM is used for authentication only (giving windows credentials to the web application) then Crowd is used to store and retrieve user's authorizations.
Atlassian Crowd provide full spring-security integration, and a bunch of connector to communicate with most of directory services available.
I first configured Crowd to use an internal directory to avoid being disrupted by communication with external directories. Then I added connection to our Microsoft Active Directory... but this go beyond the scope of this blog entry.

Prerequisites

Add spring-security and spring-security-ntlm and theirs dependency in your project
Add crowd-integration-client and his dependency in your project
Integrate Crowd integration client by following this documentation from step 1 to 3.0

Integration
First of all, create a new spring application context (applicationContext-security.xml) with the following content :

<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:sec="http://www.springframework.org/schema/security" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-2.0.2.xsd"> <sec:authentication-manager alias="authenticationManager"/>  <bean id="daoAuthenticationProvider" class="com.mycompany.security.UserDetailsAuthenticationProvider"> <sec:custom-authentication-provider/> <property name="userDetailsService"> <ref local="crowdUserDetailsService" /> </property> </bean>  <bean id="crowdUserDetailsService" class="com.atlassian.crowd.integration.springsecurity.user.CrowdUserDetailsServiceImpl"> <property name="authenticationManager" ref="crowdAuthenticationManager" /> <property name="groupMembershipManager" ref="crowdGroupMembershipManager" /> <property name="userManager" ref="crowdUserManager" /> <property name="authorityPrefix" value="ROLE_" /> </bean>  <bean id="ntlmFilter" class="org.springframework.security.ui.ntlm.NtlmProcessingFilter"> <sec:custom-filter position="NTLM_FILTER" /> <property name="stripDomain" value="true" /> <property name="defaultDomain" value="PMINTL.NET" /> <property name="domainController" value="pmichneugc01.pmintl.net" /> <property name="authenticationManager" ref="authenticationManager" /> </bean> <bean id="ntlmEntryPoint" class="org.springframework.security.ui.ntlm.NtlmProcessingFilterEntryPoint"> <property name="authenticationFailureUrl" value="/denied.jsp" /> </bean> <bean id="exceptionTranslationFilter" class="org.springframework.security.ui.ExceptionTranslationFilter"> <property name="authenticationEntryPoint" ref="ntlmEntryPoint" /> </bean>  <sec:http auto-config="true" entry-point-ref="ntlmEntryPoint" servlet-api-provision="false"> <sec:intercept-url pattern="/secure.jsp" access="ROLE_myapp-users" /> <sec:intercept-url pattern="/admin.jsp" access="ROLE_myapp-administrators" /> <sec:intercept-url pattern="/**" filters="none" /> </sec:http> </beans>

Then add it in your web application descriptor (web.xml) with the spring security filter, you must end with something like this :

And finally, create the custom UserDetailsAuthenticationProvider (unfortunally I was suprised to didn't find a provider that suit my needs...):

package com.mycompany.security; import org.springframework.dao.DataAccessException; import org.springframework.security.AuthenticationException; import org.springframework.security.AuthenticationServiceException; import org.springframework.security.providers.UsernamePasswordAuthenticationToken; import org.springframework.security.providers.dao.AbstractUserDetailsAuthenticationProvider; import org.springframework.security.userdetails.UserDetails; import org.springframework.security.userdetails.UserDetailsService; public class UserDetailsAuthenticationProvider extends AbstractUserDetailsAuthenticationProvider { private UserDetailsService userDetailsService; @Override protected void additionalAuthenticationChecks(final UserDetails userDetails, final UsernamePasswordAuthenticationToken authentication) throws AuthenticationException { // TODO Auto-generated method stub } public UserDetailsService getUserDetailsService() { return userDetailsService; } @Override protected UserDetails retrieveUser(final String username, final UsernamePasswordAuthenticationToken authentication) throws AuthenticationException { UserDetails loadedUser; try { loadedUser = this.getUserDetailsService().loadUserByUsername(username); } catch (final DataAccessException repositoryProblem) { throw new AuthenticationServiceException(repositoryProblem.getMessage(), repositoryProblem); } if (loadedUser == null) throw new AuthenticationServiceException("User cannot be null"); return loadedUser; } public void setUserDetailsService(final UserDetailsService userDetailsService) { this.userDetailsService = userDetailsService; } }

You're done !

Creating test pages
As you may have notified, the file applicationContext-security.xml defining three different level of security:

Anonymous access authorized by default to all page
<sec:intercept-url pattern="/**" filters="none" />
Standard users can access the 'secure.jsp' page
<sec:intercept-url pattern="/secure.jsp" access="ROLE_myapp-users" />
Administrators can access the 'admin.jsp' page
<sec:intercept-url pattern="/admin.jsp" access="ROLE_myapp-administrators" />

So create the pages index.jsp, secure.jsp and admin.jsp with the content you want or change the security context to suit your needs.

Crowd configuration
Open the Crowd administrator console, and create user corresponding to your Active Directory account name, add it to the application you created when you integrated Crowd (more informations).
Create two groups (myapp-user and myapp-administrators, you can name it differently but you must then update your spring security context correspondingly) and add them to the user created before.
Don't hesitate to read the Crowd documentation for further information. Specially if you want add a connector for your directory, but I strongly recommend you to first test your application using a Crowd integrated directory.

Conclusion
Integrating spring-security into an existing web-application is relatively straight-forward, specially if you already using spring. Some specific configuration is needed to enable NTLM and Crowd support but documentation can be easily find online.
And best of all, if you want to change for Kerberos in future you can do it simply and without changing the way your application works... thanks to Spring IoC :-D
And thanks to Crowd you can change connection to different directory services without loosing user authorizations (of course user-name must be the same...).

References