This site exploits technical cookies for improving navigation. No cookies are used for profiling you for commercial purpouses. Clicking the OK button you accept their usage to the end of this site navigation.  Read More
^
April 19, 2017
Posted by Claudio Guidi
Here we host a post from Danilo Sorano who collaborated with us and the Jolie team during his training period at Imola Informatica and italianaSoftware. The training period of Danilo is part of his studies at the University of Bologna. Danilo chose to work on a project about Jolie and databases. In particular, he contributed to develop a tool for automatically extracting a jolie service which facilitates the interactions with an existing database in PostgreSQL.

Congratulations to Danilo! His post follows:

JDM
Jolie Database Manager for PostgreSQL database by Danilo Sorano

The jolie database manager is a tool whose goal is to create a facilitator for the management of PostgresSQl data sources by using microservices. The tool can be get on github: https://github.com/jolie/db_connector
The main purpose of JDM is to simplify database management operations, specifically the simplifications of the insertion, modification, deletion and retrieval of data. The tool has been designed for preventing the user to write standard queries on tables and views which can be easily automatized.





Usage instructions:
  1. Creating the database and tables / views. Only if you are creating a database from scratch. Otherwise skip this step. 
  2. Start the JDM server running main_table_generator.ol inside folder server
  3. Configure the information necessary to connect with the database (file config.ini)
  4. Start the client running createDbService.ol inside folder client
  5. In folder db_services all the files will be generated: Metadata extraction from the database and creation of the service for the database.
The database service is divided in two parts:
  • Automatic service
  • Custom service
The automatic part provides basic operations for the database:
  • Create: INSERT query
  • Update: UPDATE query
  • Remove: DELETE query
  • Get: SELECT query
The management of more complex operations such as JOIN between tables must be managed by creating view by the user.

The custom part allows for  the development of customized query on the database and it can be freely edited by the user. This function gives to the user the possibility of being able to create the most complex operations, for example, to manage nested query.

From an architectural point of view, the database service's main part  embeds the custom part and automatic one. This mechanism allows the two parties to be independent of each other, thus allowing the modification of the automatic part every time that, for example, is added to a view without going to change the custom part.

Example: Creation of a database service In order to explain how the tool works we will use the example contained in the folder "examples", where there is a file sql "e-commerce.sql" to be used to generate the test database. The file creates a database of a simple e-commerce with three tables. The three tables are:
  • user (fiscal code, name, surname, email)
  • product (bar code, product name, description, quantity)
  • order (Order id, product id, user id, quantity)
The service can be used by changing the information in the file "config.ini” [db_connection] HOST=localhost DRIVER=postgresql PORT=5432 DATABASE=e-commerce USERNAME=postgres PASSWORD=postgres
Creating a service for a database
  1. Creation of PostgreSql database and its tables and views (It's possible using the e-commerce example.
  2. Starting the Tool for generate the service (main_table_generator.ol)
  3. Change the information in the config.ini file.
WINDOWS
  • Start the script server TableGenerator.bat
  • Start the script client CreateDbService.bat
  • If everything went well the service is created
OTHER OS
  • Go in the folder "server" and run the jolie file "main_table_generator.ol
  • Go in the folder "client" and run the jolie file "createDbService.ol
  • If the creation of the service is successful, the server print the message: “Database table generation is finished with SUCCESS”.
Using the service of the exampleThe generated service for the example of the e-commerce is divided into two parts:
  • automatic
  • custom
The automatic part is represented by the file main_automatic_ecommerce.ol which contains the operations:
  • createuser, createproduct and createorder
  • getUser, getproduct and getOrder
  • updateUser, updateproduct and updateorder
  • removeuser, removeproduct and removeorder.
On the other hand, the custom part as mentioned above, allows the user to create his own customized operations. Example of a clientFile try_operations.ol in the example, shows the usage of the basic operations of the automatic part. The file is structred as it follows:
  1. Importing of the interface and the location
    include"../db_services/e_commerce_handler_service/automatic_service/public/interfaces/includes.iol"
    include "../db_services/e_commerce_handler_service/locations.iol"
  2. Creation of an output port for the service
    outputPort Test {
    Location: e_commerce
    Protocol: sodep
    Interfaces: userInterface, orderInterface, productInterface
    }
    Note that the default location is "socket://localhost:9100" taht is contained inside the file locations.iol
  3. Inside the main you can make the call to the service operations. 
The FilterBefore introducing the individual operations it is better to explain the filter field. The filter field is a FilterType type which is defined inside "automatic_service/public/types/e_commerceDatabaseCommonTypes.iol" type ExpressionFilterType: void { .eq?: bool .gt?: bool .lt?: bool .gteq?: bool .lteq?: bool .noteq?: bool } type FilterType: void { .column_name: string .column_value: any .expression: ExpressionFilterType .and_operator?: bool .or_operator?: bool } The filter field is important to define the where clause of the query:.
  • column_value: value of the colum
  • column_name: name of the column of the expression
  • expression: the operator used to check the value, this is an ExpressionFilterType
  • and_operator: we define this field only if we want to concatenate another expression, in this case an AND operator
  • or_operator: we define this field only if we want to concatenate another expression, in this case an OR operator
The ExpressionFilterType define the operator:
  • lteq: "<="
  • eq: "="
  • gt: ">"
  • lt: "<"
  • gteq: ">="
  • noteq: "!="
As an example, let us suppose to define the following where clause "WHERE 'product_name' = 'Fried Chicken' AND quantita >= 10": getproductRequest.filter.column_name = "product_name"; getproductRequest.filter.column_value = "Fried Chicken"; getproductRequest.filter.expression.eq = true; getproductRequest.filter.and_operator = true; getproductRequest.filter[1].column_name = "quantity"; getproductRequest.filter[1].column_value = 10; getproductRequest.filter[1].expression.gteq = true; Example of a Get operationTo select one or more row in the product table of the e-commerce example we use the operation getproduct getproduct( getproductRequest )( getproductResponse ) throws SQLException SQLServerException It is worth noting that the fields inside the request type is only the filter fields. type getproductRequest:void { .filter*:FilterType } In the response message we will get the rows of the select query type getproductRowType:void { .id_product:int .product_name:string .description:string .quantity:long } type getproductResponse:void { .row*:getproductRowType }This operation select one or more row inside the product table. Here an example of the call: /*Get of the all rows of “Fried Chicken” with the quantity equal or greater then ten*/ getproductRequest.filter.column_name = "product_name"; getproductRequest.filter.column_value = "Fried Chicken"; getproductRequest.filter.expression.eq = true; getproductRequest.filter.and_operator = true; getproductRequest.filter[1].column_name = "quantity"; getproductRequest.filter[1].column_value = 10; getproductRequest.filter[1].expression.gteq = true; getproduct@Test(getproductRequest)( response );Example of a Create operationTo insert a row in the user table we use createuser operation createuser( createuserRequest )( createuserResponse ) throws SQLException SQLServerException - createuserRequest The fields inside the request type are the fields of the user table. type createuserRequest:void { .fiscalcode:string .name:string .surname:string .email:string } - response: createuserResponse it's a void type This operation insert a row inside the user table, to understand better how it’s work we show an example: /*The user John Silver is inserted inside the user table*/ createuserRequest.fiscalcode = "1"; createuserRequest.name = "John"; createuserRequest.surname = "Silver"; createuserRequest.email = "john.silver@mail.com"; createuser@Test(createuserRequest)(); Example of a Remove operationTo insert a row in the user table we use removeuser operation removeuser( removeuserRequest )( removeuserResponse ) throws SQLException SQLServerException - removeuserRequest There is only one filter fields, where specify the condition of WHERE clause type removeuserRequest:void { .filter*:FilterType } - response: removeuseResponse it's a void type This operation remove one or more row inside the user table, to understand better how it’s work we show an example: /*In this case we remove the user John Silver*/ removeuserRequest.filter.column_name = "surname"; removeuserRequest.filter.column_value = "Silver"; removeuserRequest.filter.expression.eq = true; removeuser@Test(removeuserRequest)(); Example of an Update operationTo update one or more rows in the user table we use updateuser operation updateuser( updateuserRequest )( updateuserResponse ) throws SQLException SQLServerException - updateuserRequest The fields inside the request type are the fields of the user table and filter field type updateuserRequest:void { .fiscalcode?:string .name?:string .surname?:string .email?:string .filter*:FilterType } - response: updateuserResponse it's a void type This operation update one or more row inside the user table, to understand better how it’s work we show an example: /*Changing the surname of John from “Silver” to “Smith”*/ updateuserRequest.surname = "Smith"; updateuserRequest.filter.column_name = "surname"; updateuserRequest.filter.column_value = "Silver"; updateuserRequest.filter.expression.eq = true; updateuser@Test(updateuserRequest)(); Future features
  • Ability to integrate the service, for more Database (Postgres, MySql ecc...).
  • Possibility to extend the management of most types and especially more complex.
Contact
  1. C


April 18, 2017
Posted by Claudio Guidi

As we described here, with Jolie we are pioneering the linguistic approach for dealing with microservices. Our idea is that microservices are introducing a new programming paradigm which can be crystallized within a programming language. The focus of this post is about the definition of microservice starting from our point of view: a linguistic point of view.

A service is the single unit of programmable software
In the last years the concept of service has been investigated in the area of Service Oriented Computing and several definitions have been provided for defining service contracts, service providers, service discovery and so on. All these definitions are quite abstract because services have been conceived to be technology agnostic both in the case of SOA and microservices. Such a fact means that it is possible to develop a service in any given technology. They say that services are technology agnostic.

Technology agnosticism is a very important feature which allows us to engineer a software system independently from any technology lock-in. But our purpose here is to give the definition of a service as a single unit of programmable software which cannot be fragmented in sub-parts. For this reason, you will find that all the definitions I am going to provide here are strongly related to a specific technology: Jolie. If you like to see how we chose to model the service oriented programming paradigm in a single language you can continue to read, otherwise you can skip this post. If we are wrong, or we are missing some points, or you know other technologies which match the definitions please write us your feedbacks. We are very exited to share ideas on this open topic.

As a starting point, let me explain the first assumption we made in the linguistic paradigm: the service is the single unit of programmable software. Usually, a service is always obtained by programming a server (it is not important if it is simple or not) joint with some business logic which represent the functionalities to serve:

                             SERVER + BUSINESS LOGIC = SERVICE

In a linguistic paradigm such an equation is not more valid just because servers do not exist. Only services exist. It is not possible to program a server because you can program only services. So, forget servers (do not confuse with serverless, it is a different approach). So, if there are no servers but only services, what is a service? As it happens for Object Orientation where classes are logical definitions and objects are the instances of classes in a running environment, let me call the logical definition of services with the term service and its running instance with the term microservice.

                                     SERVICE --> MICROSERVICE

For each service there could be more than one microservices, but each microservice is just the running instance of a service. The service is the single unit of programmable software. In the following I am going to build the definition of a service by giving some qualities it has to provide. At the end of this post I'll give the definition of service.

Services exchange messages
The only way for exchanging data among services are messages. There are no other way. A message is just a limited portion of data transmitted in a limited portion of time. A service can both receive messages and send messages. In a SOA a service which receives and sends messages is usually called orchestrator. Such a difference in the linguistic paradigm does not exist, an orchestrator is just a service. In particular, in Jolie message exchange is enabled by means of ports. Messages are received by means of inputPorts and they are sent by means of outputPorts. Similar constructs are used in WS-BPEL, they are called partnerLinkTypes.

Services can have a behaviour
The behaviour defines the business logic to be executed when a message is received.  In the behaviour it is possible to compute the received data and/or send messages to other services. In Jolie the behaviour is expressed in scope main. A behaviour can define different operations. Different business logics can be joint to different operations. In Jolie multiple operations can be expressed in the same behaviour by using the non deterministic operator:

main {
[ op1( req1 )( res1 ) {
    businessLogic1
}]  

[ op2( req2 )( res2 ) {
    businessLogic2
}]  

...

[ opn( reqn )( resn ) {
    businessLogicn
}]  
}

An operation must always express a finite computation in time. In other words, when triggered an operation must always reach an end state. I say that an operation is divergent if its behaviour defines an infinite computation. Jolie allows for the definition of divergent operations by defining infinite loops:

divergentOperation( request )( response ) {
    while( true ) {
           nullProcess 
    }
}

Divergent operations are deprecated in Jolie.

Services declare interfaces
The operations of the behaviour must be declared in a machine readable interface. The interface defines all the available operations of a given service. Interfaces are used as a concrete contract for interacting with a service. In Jolie interfaces are also equipped with message type declarations.

type MyRequestType: void {
    .msg: string
}

type MyResponseType: void {
    .resp_msg: string
}

interface myInterface {
RequestResponse:
     myOperation( MyRequestType )( MyResponseType )
}

Services execute sessions
Services execute sessions for serving requests. A session is a running instance of a service operation. Sessions are independently executed and their data are independently scoped. If we suppose to send three messages to a microservice which implements the following service we will receive three different messages for each request message.

main {
   test( request )( response ) {
      response = request
   }
}

if we concurrently send three request messages with content "1", "2", "3", we will receive three related reply messages with the same content.


A definition of service

Here I try to summarize a definition of service starting from the basic concepts highlighted above. Jolie provides more linguistic elements w.r.t. those highlighted here. Maybe more basic concepts could be added to the definition and other should be removed. This is just a starting point for trying to investigate microservices from a linguistic point of view. All the contributions are encouraged!

A service is a unit of programmable software able to exchange messages with other services whose behaviour is triggered by incoming messages and it is defined by a set of finite computation logics called operations which are declared within a machine readable interface. A running instance of a service is called microservice whose inner instances of triggered operations are called sessions. Sessions are executed independently and their variables are independently scoped.









December 29, 2016
Posted by Claudio Guidi
Il 20 Dicembre si è svolto a Bologna il Meeting on Microservices (MoM) organizzato da italianaSoftware al fine di creare un primo momento di incontro tra aziende, pubbliche amministrazioni e mondo della ricerca sul tema dei microservizi. Siamo molto contenti che l'evento abbia avuto successo ed abbia visto la partecipazione di un buon numero di interessati ed una serie di interventi davvero stimolanti. Come prima conclusione a seguito di questa giornata, si può sicuramente affermare che il tema "microservizi" è ad oggi un tema di assoluto interesse e che tutti gli addetti ai lavori sono intenzionati a conoscere e a comprendere maggiormente le opportunità ed i rischi che questo nuovo approccio architetturale può offrire.


L'elemento positivo della prima edizione del MoM è certamente quello di essere riuscito a mettere insieme punti di vista sia teorici ed accademici che casi pratici d'uso di quelle aziende che per prime hanno implementato soluzioni orientate ai microservizi. La cosa probabilmente più rilevante che qui ci preme mettere in risalto è come questa soluzione architetturale possa essere adottata da aziende di tutte le dimensioni per risolvere i problemi più disparati. Le presentazioni di Monrif ed H2B hanno illustrato come si possano gestire con i microservizi sia i flussi di archiviazione documentale che l'implementazione di una vera e propria piattaforma B2B. Sicuramente va sottolineato come la loro adozione necessita di ripensare alcuni dei processi esistenti per la progettazione e la programmazione del software e che tante sfide si aprono ora nel mondo IT a seguito dell'adozione dei microservizi. Da una parte l'analisi ed il design dell'architettura diventano passi fondamentali per tutti i team IT che intendono adottare i microservizi e dall'altra molte attività che vengono generalmente esternalizzate possono ora essere riportate all'interno producendo da una parte risparmio e dall'altra una risposta più celere alle esigenze del business.

Il dibattito sui microservizi è appena entrato nel vivo e nei prossimi anni siamo certi che si riparlerà a più riprese di essi. Noi, come italianaSoftware ci saremo e lavoreremo affinchè il MoM possa diventare un evento periodico di carattere nazionale che possa diventare un riferimento per tutti coloro che sono interessati all'argomento.




Di seguito riportiamo l'elenco delle presentazioni con un brevissimo riassunto sui loro contenuti.

Genesi di una tecnologia, dalla ricerca all'industria... - Maurizio Gabbrielli (DISI - Università di Bologna)
Innovare è una sfida sempre difficile dove la possibilità di sbagliare deve essere considerata sin dall'inizio. Lo sviluppo e la maturazione della tecnologia Jolie sono un caso di studio interessante dove dallo sviluppo di un modello matematico si è riusciti a passare ad un linguaggio di programmazione che porta benefici al mondo produttivo.



La rivoluzione dei microservizi - Claudio Guidi (italianaSoftware)
I microservizi promettono di rivoluzionare il modo di progettare e programmare software. italianaSoftware propone una tecnologia specifica per affrontare la programmazione dei microservizi: jolie. Essa si distingue perché propone di approcciare i microservizi tramite un paradogma linguistico piuttosto che tramite un paradigma sistemistico che è quello più conosciuto fino a questo momento.



Implementazione di una soluzione a microservizi: benefici organizzativi ed economici - Balint Maschio (Monrif s.p.a.)
Monrif s.p.a. ha iniziato ad utilizzare soluzioni a microservizi inizialmente per risolvere i problemi di system integration legati all'archiviazione documentale da ambiente SAP verso applicativi di terze parti. Ora ha iniziato a creare veri e propri flussi tramite micro-orchestratori dedicati.


Industria 4.0, come verrà rivoluzionata l'industria italiana - Paola Perini (Innovami)
Industria 4.0 è la nuova parola chiave con la quale si vuole descrivere il processo già in atto di rivoluzione del settore industriale italiano dove "ogni azienda si dovrà trasformare anche in un'azienda software".



H2B e i microservices: un caso di successo - Alessandro Suzzi (H2B)
I microservizi hanno aiutato H2B nel realizzare sia il loro applicativo ad agenti che i portali web B2B per la gestione del catalogo prodotti e degli ordini per un insieme di aziende nel settore delle ferramente gestite da H2B.

Dalle Service Oriented Architectures (SOA) ai microservizi - Claudio Bergamini (Imola Informatica)
Service Oriented Architectures e microservizi vengono spesso accoumnati e confrontati tra di loro. Il tema è sicuramente interessante, entrambe posano le proprie radici sull'idea di servizio come componente seppure con differenze implementative. Di certo c'è che sarà l'analisi specifica dei diversi scenari di utilizzo a stabilire dove sia meglio adottare l'una o l'altra soluzione.

Devops, Cloud e Container - Luca Acquaviva (Imola Informatica)
L'introduzione dei microservizi e la gestione dei loro container in particolare, porta a dover ripensare la propria infrastruttura ed i propri processi Devops. La stessa architettura di un applicativo web può arrivare a cambiare radicalmente.

Microservizi, scenari del prossimo e del lontano futuro - Saverio Giallorenzo (DISI - UNiversità di Bologna)
I microservizi sono con ogni probabilità il trampolino di lancio verso scenari futuri estremamente interessanti da esplorare. Tra le diverse ramificazioni a cui essi possono portare c'è sicuramente quello delle coreografie attualmente esplorato all'interno dell'Università di Bologna.
November 12, 2016

TL;DR: decorators for services are pretty easy to write in Jolie, thanks to a native feature called aggregation. Be careful in the deployment phase for managing performance hits!

Service Decorators

Just read this article about using decorators over inheritance in object-oriented programs. I assume that you know decorators in the following. If you don’t, get informed (e.g., by reading that article) before proceeding.

The Decorator design pattern offers a way to cleanly modify the behaviour of an object by using composition instead of inheritance. I won’t enter in the merits of composition against inheritance in object-orientation (both have their own, depending on the other features of the language), because I’m interested in microservices here. In microservices, composition is typically the only option you have. That’s because your microservices may be written in different languages, or even paradigms. And even if their codebases were somehow compatible, inheritance would still be out of place: your microservices can only depend on the communication APIs of other services, not on their implementation details (e.g., by looking at their source code).

Since decorator acts through composition, this pattern is potentially interesting for service programming. But as the author of the article linked above mentions, using it can be frustrating since it requires a lot of boilerplate code. This makes it also error-prone. As a running example, let’s look at the e-mail service interface proposed in that article, rewritten in Jolie:

interface EmailServiceIface {
RequestResponse:
  send(Email)(void),
  listEmails(Range)(EmailInfoList),
  downloadEmail(EmailInfo)(Email)
}

Decorating an operation

Now, suppose that this service is available to us as EmailService. Assume also that we want to write a new service that decorates EmailService with the following logic: whenever send is called, we check if we have sent an “important” e-mail (e.g., the subject contains a specific keyword telling us that the e-mail is important, or the addressee is in a special list); if an important e-mail has been sent successfully, we backup its content by calling another service, for indexing and safe-keeping.

A naive implementation is to write the decorator by re-implementing interface EmailService, like this:

inputPort EmailServiceImportant {
Location: MyLocation Protocol: MyProtocol
Interfaces: EmailServiceIface
}

main
{
  [ send( request )( response ) {
    send@EmailService( request )();
    check@Important( { .subject = request.subject, .to = request.to } )( important );
    if ( important ) {
      backup@BackupService( request )()
    }
  } ]

  [ listEmails( request )( response ) {
    listEmails@EmailService( request )( response )
  } ]

  [ downloadEmail( request )( response ) {
    downloadEmail@EmailService( request )( response )
  } ]
}

The code for listEmails and downloadEmail is boilerplate, since we’re just forwarding requests and responses. The author of the article suggests that it would be nice if languages supported a native feature that makes writing this code unnecessary. Luckily, we have it in this case!

Forwarding is a native feature in Jolie, since building proxies is the bread and butter of service composition (think of load balancers, caches, etc.). We can rewrite our decorator like this:

inputPort EmailServiceImportant {
Location: MyLocation Protocol: MyProtocol
Aggregates: EmailService // Aggregates instead of Interfaces!
}

main
{
  [ send( request )( response ) {
    send@EmailService( request )();
    check@Important( { .subject = request.subject, .to = request.to } )( important );
    if ( important ) {
      backup@BackupService( request )()
    }
  } ]
}

No boilerplate, same behaviour as our previous decorator!

Interface Decoration

What if you want to change the behaviours of many operations, not just one? What if this behaviour change is always the same? For example, suppose that we want to write a decorator that keeps track of all events: whenever an operation is called, we write this in an external log.

Here’s a naive implementation:

inputPort EmailServiceLogger {
Location: MyLocation Protocol: MyProtocol
Interfaces: EmailServiceIface
}

main
{
  [ send( request )( response ) {
    send@EmailService( request )();
    log@Logger( request )()
  } ]

  [ listEmails( request )( response ) {
    listEmails@EmailService( request )( response );
    log@Logger( request )()
  } ]

  [ downloadEmail( request )( response ) {
    downloadEmail@EmailService( request )( response );
    log@Logger( request )()
  } ]
}

Ouch, boilerplate again! What we need is the capability of writing that logging code just once, and applying it to the entire API of EmailService. That’s what courier processes in Jolie are for. Here’s the improved code, using courier:

inputPort EmailServiceLogger {
Location: MyLocation Protocol: MyProtocol
Aggregates: EmailService // Aggregates again
}

courier EmailServiceLogger // courier enables primitives for whole-interface behaviour
{
  [ interface EmailServiceIface( request )( response ) ] { // Apply to all operations in the interface
    forward( request )( response ); // forward is a primitive: it forwards the message to the aggreated (in this case, decorated) service
    log@Logger( request )()
  }
}

No boilerplate again!

Circuit breakers

What’s a cool example of a decorator? Circuit breaker! Yup. A sketch in Jolie using aggregation can be found in this paper.

Conclusions

As cool as decorators are, don’t forget that you are adding a layer of indirection. Many times, you don’t have a choice and your code benefits so much that you should pay the price. But in microservices, be very careful about how efficient your extra layer will be. If you stack too many decorators and they are all communicating remotely via sockets, you’ll soon have a performance problem. This is a deployment challenge: in Jolie, using a different communication medium doesn’t alter your behavioural code. So choose your communication media wisely when you deploy! If you have control over your stack of decorators, consider whether it would be better to put them all in one place and have them communicate using shared-memory (local location in Jolie).

October 12, 2016
Posted by Claudio Guidi
Jolie is now available on Docker and now it is possible to develop and run a microservice inside a container.

But what about the deployment of a microservice in Docker? How can we build a deployable docker container which includes the microservice we are working on?

Actually, it is a very easy task. It is sufficient to develop the microservice by following some simple rules and your docker image will be ready in few seconds!

Rule 1 : you need a Dockerfile for building an image of your microservice
First of all, create a file named Dockerfile in your working directory and write the following lines:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME


where SURNAME, NAME and EMAIL must be replaced with the maintainer's surname, name and email respectively. The Dockerfile will be used by Docker for building the image related to your microservice. As you can see, the image you are creating is layered upon a previously created image called jolielang/jolie-docker-deployer.
You can find this image in the docker hub of jolielang here. Such a docker image, is prepared for facilitating the deployment of a jolie microservice as a docker image. In order to use it in the right way, just follow the next rules. As an example, let me suppose to deploy the following microservice saved in file helloservice.ol:

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

execution{ concurrent }

inputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

main {
  hello( request )( response ) {
        response = request
  }
}


Rule 2 : EXPOSE inputPorts ports
Remember that all the inputPorts of your microservice must be reached from outside the container, thus you need to expose them in the Dockerfile.



In the example the inputPort is located at localhost:8000, thus we need to add EXPOSE 8000 in the Dockerfile. So, your Dockerfile now becomes like this:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8000

Rule 3 : COPY the files of your project and define the main.ol
Now, everything is quite done for preparing the image, we just need to copy the files of the project in the docker image. When doing this, pay attention to rename the file which must be run with the name main.ol.

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8000
COPY helloservice.ol main.ol

Building your image

When the Dockerfile is ready we can build the docker image of the microservice. In order to do this you just need to run the following command within your working directory which also contains the Dockerfile.

docker build -t hello .

where hello is the name we give to the image. Once it is finished, you can easily check the presence of the image in the local registry by running the following command:

docker images

Running a container
Now, starting from the image, you can run all the containers you want. A container can be run by launching the following command:

docker run --name hello-cnt -p 8000:8000 hello

where hello-cnt is the name we give to the container. Note that the parameter -p allows you to map the microservice port (8000) to the port 8000 of your localhost. You can check that the container is running just launching the following command which lists all the running containers:

docker ps

Your microservice is now deployed and it is listening for requests at port 8000. You can just try to invoke it with a client like the following one. Remember to launch the client in a separate shell of your localhost!



include "console.iol"

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}


outputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

main {
  hello@Hello( "hello" )( response );
  println@Console( response )() 
}



Advanced settings
So far, we have deployed a very simple service but, usually we deal with microservices that are more complicated than the hello service presented before. In particular, it is very common the case where some constants or outputPort locations must be defined at deploying time. In order to show this point, let me now consider the following service:

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

execution{ concurrent } 

constants {
   CUSTOM_MESSAGE = " :plus!"


outputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

inputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface 
}

main {
  helloPlus( request )( response ) {
        hello@Hello( request )( response );
        response = response + CUSTOM_MESSAGE
  }


This is a very simple microservice which has a dependency on the previous one. Indeed, in order to implement its operation helloPlus, it requires to invoke the operation hello of the previously deployed microservice. Moreover, it uses a constants CUSTOM_MESSAGE for defining a string to be added to the response string.




Usually, we would like that some of these parameters can be defined at deploying time because they directly deal with the architectural context where the microservice will run. Thus, we would like to create an image which is configurable when it is run as a container. How can we achieve this?

Rule 4 : Prepare constants to be defined at deploying time
The image jolielang/jolie-docker-deployer we prepared for deploying microservice has been built with some specific scripts which are executed before running the main.ol. These scripts just read the environment variables passed to the docker container and transform them in a file of constants which must be read by your microservice. The most important facts we need to know here are:
  1. Only the environment variables prefixed with JDEP_ will be processed
  2. The processed environment variables will be collected in a file of constants  named dependencies.iol. If it exists it will be overwritten.
From these two points we change the microservice as it follows:
 

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

include "dependencies.iol"

execution{ concurrent } 

outputPort Hello {
Location: JDEP_HELLO_LOCATION
Protocol: sodep
Interfaces: HelloInterface
}

inputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface 
}

main {
  helloPlus( request )( response ) {
        hello@Hello( request )( response );
        response = response + JDEP_CUSTOM_MESSAGE
  }
}

As you can notice here there are two constants JDEP_HELLO_LOCATION and JDEP_CUSTOM_MESSAGE which require to be defined at the start of the microservice. They must be defined in the file dependencies.iol which MUST BE included in your microservice. This file just contains the declaration of the two constants.

constants {
JDEP_HELLO_LOCATION = "socket://localhost:8000",
JDEP_CUSTOM_MESSAGE = " :plus!"
}

During the development keep this file in your project and collect here all the constants you want to define at deploying time. When the service will be run in the container this file will be overwritten thus, you don't need to copy it into the docker image.

The Dockerfile of the helloPlus service is very similar to the previous one:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8001
COPY helloservicePlus.ol main.ol
We can create the image with the same command used before, but where the name is hello-plus.

 docker build -t hello_plus .

Configuring the container
Now, we just need to know how to pass the constants to the running container and everything is done. Docker allows to pass environment variables with the parameter -e available for command run. Thus the command is:

docker run --name hello-plus-cnt -p 8001:8001 -e JDEP_HELLO_LOCATION="socket://172.17.0.4:8000" -e JDEP_CUSTOM_MESSAGE=" :plus!" hello_plus

where hello-plus-cnt is the name we give to the container. Note that the constant JDEP_HELLO_LOCATION is set to  "socket://172.17.0.4:8000" where the IP is set to 172.17.0.4. It is just an example, here you need to specify the IP that Docker assigned to the container hello-cnt which is executing the service hello.ol. You can retrieve it just launching the following command:

docker inspect --format '{{ .NetworkSettings.IPAddress }}' hello-cnt

Once the hello-plus-cnt container is running, you can simply invoke it with the following client:

include "console.iol"

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

outputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface
}

main {
  helloPlus@HelloPlus( "hello" )( response );
  println@Console( response )()
}


Conclusion
In this post I show how it is possible to deploy a microservice developed with Jolie as a docker container. The procedure is very easy, just pay attention to inputPorts and the constants you want to configure at deploying time. For all the other things you can just rely on the Jolie language. Don't forget that you can also exploit embedding for packaging more microservices into one, thus deploying all of them inside the same container if necessary.

Enjoy!

September 05, 2016
Posted by Claudio Guidi
This post is about a couple of tools [https://github.com/jolie/jester] we developed for facilitating the programming of REST microservices with Jolie. Personally I am not a big supporter of REST services, but I think that a technology which aims at being a reference in the area of microservices like Jolie must have some tools for supporting REST services programming. Why? Because REST services are widely adopted and we cannot ignore such a big evidence.

Ideally, Jolie as a language is already well equipped for supporting API programming also using http, but REST approach is so deep coupled with the usage of the HTTP protocol that it introduces some strong limitations in the service programming paradigm. Which ones? The most evident one is that a REST service only exploits four basic operations: GET, POST, PUT and DELETE. The consequence of such a strong limitation on the possible actions is that the resulting programming style must provide expressiveness on data. This is why the idea of resources has been introduced in REST! Since we cannot programming actions we can only program resources.

Ok,  let's go with REST services!

...But,
...here we have a language, Jolie, that is more expressive than REST because the programmer is free to develop all the operations she wants. From a theoretical point of view, in Jolie she can program infinite actions instead of only four!



- Houston we have a problem! We need to put infinite operations inside just four!!!

- Why only four when we can have infinite?

- This does not matter now Houston, we have to do it!



Ok no problem! Follow our instructions!
First of all, take note that here we want to preserve all the benefits of using Jolie, thus the possibility to develop all the operations we want, but finally publish the microservice as a REST service. We achieve such a result by exploiting a specific microservice architetcure which is a composition of a router and the target microservice we want to publish as a REST one. The router, as described in Fabrizio's paper [https://arxiv.org/abs/1410.3712], is in charge to transform REST calls into standard Jolie calls.



Ok, in order to exaplain how to proceed, let us consider as a target microservice the demo one reported in the jester project [https://github.com/jolie/jester/tree/master/src/jolie/tools/demo]. It is a very simple service which emulates a manager of orders which supplies four operations: getOrders, getOrdersByItem, putOrder, deleteOrders.

Ok, there are four operations but it is just an example, we could have more than four operations :-). The main question now is:

What we have to do for transforming these operations in REST operations?
We need to use the jolie2rest.ol tool you can find the jester project. Such a tool analyzes the interface of the demo service and it extracts a descriptor for the router which enables it to publish the demo service as a REST service.

Very simple! But before running the tool we need to know something more. We need to define how to convert the single operations of the demo microservice will be tranformed into the REST one. In particular, fro each target operations we need to specify if we want a GET operation, a POST operation, a PUT operation or a DELETE operation. It is possible to provide these instructions, directly into the interface of the demo service by exploting the annotation @Rest into the documentation comments. As an example let us consider the operation getOrders:

/**! @Rest: method=get, template=/orders/{userId}?maxItems={maxItems}; */
getOrders( GetOrdersRequest )( GetOrdersResponse )


The annotation defines that the operation getOrders it must be transformed into a GET http method and the URL template that must be adopted is /orders/{userId}?maxItems={maxItems}. What is the template URL?
Since the REST services deal with resources, the URL is used as a means for expressing the resource we want to access. Here we use the template as the means for transforming a call to an operation of the target service into a resource. In particular, the parameters between curly brackets will be directly filled by using the corresponding values of the request message node which is defined in the GetOrdersRequest type:

type GetOrdersRequest: void {
    .userId: string
    .maxItems: int
}


Now, we can proceed by running the following command:

jolie jolie2rest.ol localhost:8080 swagger_enable=true

where localhost:8080 is the location where the router is deployed and the parameter swagger_enable specifies if we want to enable the creation of the swagger json descriptor file.
Note: the file service_list.txt contains the list of the target microservices to be transformed and the related inputPorts to be converted (more instructions can be found here: ref).

As a result the tool will generate two files:
  • router_import.ol
  • swagger_DEMO.json
The former file must be copied in the router folder and it contains the script which enables the router to transform the REST calls into the operations call of the target service. The latter is just the swagger descriptor to be provided in input into a SwaggerUI. You don't have a local SwaggerUI available? Follow these instructions to have it locally, otherwise go to point 6:



  1. Prepare the web server of the SwaggerUI application by downloading Leonardo from here [https://github.com/jolie/leonardo]
  2. Go to this SwaggerUI URL [https://github.com/swagger-api/swagger-ui/archive/v2.2.3.zip] and download the related web application project. 
  3. Copy the content of the folder dist of the SwaggerUI project inside the folder www of Leonardo.
  4. Open a shell and run jolie leonardo.ol
  5. Open a browser at the url http://localhost:8000, the SwaggerUI web application should appear.
  6. Copy the swagger_DEMO.json file into the folder where it is reachable from the SwaggerUI, in the Leonardo scenario put it inside the www folder.
  7. Write in the explorer bar of the SwaggerUI the URL for reaching the swagger descriptor, in the Leonardo case write: http://localhost:8000/swagger_DEMO.json
After this step, you should see the Swagger definition of your target demo service transformed into a REST one. In particular, you should see something like this:







As you can see the four operations of the target service have been transformed in the four different types of REST operations. Have a look to the interface annotations of the demo service for finding the matches with the Swagger interface!

Nice, but what happen if I want to transform more than one Jolie microservices instead of a single one?
This question is dealing with an architecture like the following one where the router is connected with several microservices:



There are no particular problems in achieving such a result. It is sufficient to list all the target microservices with the related input ports in the file service_list.txt and re-launch the jolie2rest tool!

Ok Houston! Here we are! But is it possible to avoid the usage of the router?
Yes, it is possible. But only if you accept to not completely adhere to the REST approach. A jolie microservice can be directly published by using a http inputPort with message format set to json [http://docs.jolie-lang.org/#!documentation/protocols/http.html#http-parameters]. In this case the microservice will be able to serve the http requests without requiring any router or proxy in the middle. If you want this, change the inputPort protocol of the demo service to a http one and then use the jolie2rest tool with the parameter easy_interface set to true.

jolie jolie2rest.ol localhost:8080 swagger_enable=true easy_interface=true

In such a case the router_import.ol file is not generated but only the swagger_DEMO.json one. The operations are all converted in POST methods and the URLs do not follow the templates but the request json messages must be entirely defined in the body of the message. Try to replace the file swagger_DEMO.json in the SwaggerUI and perform some calls.

Generating client stubs from an existing Swagger definition:
A last tool which can be very useful when integrating existing REST services in a Jolie architecture, is jolie_stubs_from_swagger.ol. Such a tool takes in input an existing Swagger definition and generates a Jolie client for each published api.

As an example you could try it by generating the clients for the petstore example supplied by the Swagger project [http://petstore.swagger.io/v2/swagger.json]. In order to do it, create a target folder where you want to store all the generated clients (for example petstoreFolder) and then run the following command:

jolie jolie_stubs_from_swagger.ol http://petstore.swagger.io/v2/swagger.json petstoreService petstoreFolder

where petstoreService is the token that will be used for generating the Jolie outputPort name of the petstore service inside the clients. As a result, in folder petstoreFolder you will find a list of Jolie clients. In particular, you will have a client for each api defined in the petstore swagger definition.

If you want to try to send a request just open one of them and create the request message. For example open the getOrderById.ol and prepare the request message by adding the following jolie code:

with( request ) {
.orderId = 8
};

then run the client as a usual jolie script:

jolie getOrderById.ol


the result should be printed out on the console!

You can exploit these clients inside your existing jolie microservices. Just note that the generated file outputPort.iol defines all the information necessary to declare the outputPort to be used. Thus just include it in your microservice project and then make the calls when is more useful for you!

Conclusion
Houston, everything is clear now! :-) With the REST tools described in this article we want to improve the Jolie language providing the possibility to publish Jolie microservices as REST services and giving an easy way for generating clients from existing REST services. We hope this could be useful for your  projects and, please, do not forget to send us your feedbacks and improvements!





May 15, 2016

A pattern that arises often when writing a (micro)service is that of a workflow that provides access to a resource until something special happens. The new provide-until primitive in Jolie supports exactly this pattern. It doesn’t add any expressivity to the language (we could do this before using bookkeeping variables, just like in any other language), but it makes the code more readable and pleasant in this kind of situations that come up so frequently.

Here is an example of a service that handles shopping carts. We can start a process in our service by invoking operation createCart. We then set a timeout (given by the constant TIMEOUT, omitted here). When the timeout expires, the Time service (from the Jolie standard library) is going to call us on operation timeout. Now we are all set and we enter a provide-until block. Provide-until takes two input choices. The reasoning is very simple: the operations in the first input choice are provided (thus can be invoked multiple times) until any of the operations in the until block is called. Below, we can invoke the operations add and remove to add and remove, respectively, items in our shopping cart, until one of the following happens: the process timeouts (operation timeout is invoked); the user decides to delete the cart (operation delete); or, the user decide to proceed to checkout (operation checkout).

/* Interfaces and ports ... */

cset { id: Add.id Remove.id Timeout.id Delete.id Checkout.id }

main
{
  createCart( request )( cart ) {
    cart.id = csets.id = new;
    cart.user = request.user
  };

  setNextTimeout@Time( TIMEOUT { .message.id = cart.id } );

  provide
    [ add( request )( cart ) {
      cart.items[ #cart.items ] << request.item
    } ]

    [ remove( request )( cart ) {
      undef( cart.items[ request.which ] )
    } ]

  until
    [ timeout() ]

    [ delete()() ]

    [ checkout( paymentInfo )() {
      // ...
    } ]
}

Provide-until is a little thing that I enjoyed developing lately. It originally comes from the desire of handling elegantly REST processes in Jolie, but it works just as well in any similar situation. I give a longer explanation for researchers of how Jolie works with the Web in this paper: https://arxiv.org/abs/1410.3712.

June 09, 2015


Thanks to Martin's great help, Jolie 1.4 will have support for Internal Services and enhanced local locations. If you feel like testing, you can already find it in our git repository (see compile from sources).
While these features do not add any essential expressiveness to Jolie, they make it easier to program non-distributed microservice architectures (NMSA here for brevity). "Wait" - I hear you - "NMSAs?"


Non-distributed Microservice Architectures (NMSAs)

When I teach somebody (students or professionals) how to program microservices, I pay attention to following a very gradual path. The fact is, there are simply too many things to worry about when you program Microservice Architectures (MSAs). If you are not an expert, you are going to get overwhelmed by the complexity. Even if you are an expert, you may be better off with the MonolithFirst approach.

But why are Microservices so complicated? The main reason is that an MSA is typically distributed, and distributed computing is hard (see the fallacies of distributed computing). So, intuitively, it is much simpler to start with a monolith, and switch to microservices later. Unfortunately intuition is not really met by reality here. Developing a monolith does not force programmers to make their components modular and loosely-coupled enough to be smoothly ported to microservices; for example, it is way too tempting to share resources and/or develop APIs that are not suitable for message passing. Most probably, your initial monolith will end up being a SacrificialArchitecture. This is not necessarily a bad thing: many successes were built on top of sacrificing architectures. I am more worried about the fact that this is not really a smooth learning process: develop a monolith, learn something, now change everything.

NMSAs is a style where you program your system as if it were an MSA, but without the distribution. In an NMSA, Microservices are run concurrently in the same container (in our case, a Jolie-controlled JVM). It is an attempt at providing an easier setting for learners and experimenters to start with: no distributed computing means no fallacies of distributed computing. What can you learn in such an environment? A great deal, including:

  • You can learn how to develop loosely-coupled service interfaces, and how to deal with message passing in the easier setting of a perfectly-working "network".
  • You can learn how to test your services, and how to automatise testing.
  • You can learn how to monitor service execution.
The idea is to first approach the programming of microservices in this simpler ("fake", if you like) setting, and learn how to deal with distribution later. In the Jolie programming language, this path from NMSAs to MSAs is supported by the language itself.

NMSAs first...

NMSAs are created in Jolie by using embedded or internal services, i.e., services that run locally in the same container of the main service program. I will refer to such services as "local services" here. All components that you can write in Jolie are services, therefore all the architectures that you write in Jolie are service-oriented by design. This means that components cannot share state, and must interact through service interfaces based on message passing. What happens behind the scenes depends on the deployment information that you specify for your services, which is kept separate from the application logic. If the case of a local service, messages are efficiently communicated via shared memory, but without breaking the message passing abstraction given to the programmer.

Here is a toy example of a program using local services (you can of course split this in multiple files; oh, and I'm omitting what happens in case we do not find some records or files):


service Images {
    Interfaces: ImagesIface
    main {
        get( request )( img ) {
            fetch@ImageDB( request )( img )
        }
    }
}

service Products {
    Interfaces: ProductsIface
    main {
        get( request )( response ) {
            query@ProductDB
                ( "select name,desc,imageurl from products where id=:id"
                    { .id = request.id } )
                ( response )
        }
    }
}

main {
    viewProduct( request )( product ) {
        get@Products( { .id = request.id } )( product );
        if ( request.embedImage ) {
            get@Images( { .url = product.imageurl } )( product.image )
        }
    }
}

The program above is a simple service to access product information. It offers one operation, viewProduct, that clients can invoke with a message containing the id of the product they want to see (as a subnode of request). The service then invoke the internal service Products, which queries a database for some basic information and returns it. The product information contains an URL to an image depicting the product. Then, the main program checks whether the original client requested the image to be embedded in the response (useful, e.g., for mobile applications that want to minimize the number of message exchanges); if positive, the image is added to the response by fetching it from the Images service.

The main service, Images, and Products communicate using shared memory in the Jolie implementation of internal services. Communications are therefore always going to succeed, but each service has its own interface and state and the style is still message passing. Hence, internal services can be used to teach how to design a loosely-coupled architecture.

...and MSAs later

NMSAs are expressive enough to teach, e.g., how to deal with concurrency issues (each service has its own processes), message passing interfaces, and good API design. They are also fairly easy to achieve, relieving beginners from the frustrations of service deployment.

After a while, however, an NMSA that needs to scale has to evolve into an MSA, by taking some internal services and making them distributed. Distributing an internal service in Jolie is easy. Simply put, take the code of the internal service you want to distribute and execute it with a separate interpreter in another machine (Jolie Enterprise automatises this job for you in a cloud environment, but you can use the tools of your choice).

The main service now needs to be updated to refer to Images and Products as external distributed services:

outputPort Images {
Location: "socket://images.example:8080"
Protocol: sodep
Interfaces: ImagesIface
}

outputPort Products {
Location: "socket://products.example:8080"
Protocol: sodep
Interfaces: ProductsIface
}

main {
    viewProduct( request )( product ) {
        get@Products( { .id = request.id } )( product );
        if ( request.embedImage ) {
            get@Images( { .url = product.imageurl } )( product.image )
        }
    }
}

Basically, we have replaced the code for each internal service with a respective output port. The rest of the code does not change (again, a key aspect of developing services with Jolie is that deployment is kept as separate as possible from behaviour). However, now the system is distributed. Among other consequences of this, communications with services Products and Images may now fail! In practice, this means that the invocations get@Products and get@Images can throw network-related faults now, and we must account for that. If we leave the code like this, such faults are going to be automatically propagated to clients. This may be undesirable. A better strategy could be:

  • If we fail to get the product information, we immediately return an informative fault to the client.
  • If we manage to get the product information but fail to get its image, we should return at least the product information with a placeholder blank image.
We update the code accordingly, using dynamic fault handlers:



main {
    viewProduct( request )( product ) {
        scope( s ) {
            install( default => throw( ProductsCurrentlyUnavailable ) );
            get@Products( { .id = request.id } )( product );

            install( default => product.image = BLANK_IMAGE );
            if ( request.embedImage ) {
                get@Images( { .url = product.imageurl } )( product.image )
            }
        }
    }
}

There we go. Now our code accounts for network errors (actually any error, since I used the default handler, which catches any fault). There are, of course, many other things to watch out for, e.g., distributed monitoring, crash recovery, and performance. However, this small example gives already an idea of the overall methodology.

Gotcha

Although Jolie does its best in not allowing resource sharing among services (they simply cannot be shared, by the language syntax), this can always be circumvented via external side-effects. For example, two internal service may share access to the same file system. However, this can also be abstracted from: in Jolie all accesses to the file system happen through the File service from the standard library which, being a service, can also be distributed.

Conclusions

The microservices style arises out of practical needs, and although it should be used only when necessary, sometimes it is appropriate. I believe that we should find ways to guide a smoother transition from monoliths to microservices, and that finding such ways is possible. This is a rough attempt at giving a first piece of the puzzle towards building such a transition model.


PS: I should really try to come up with better acronyms.
PPS: If you have interesting examples to share in other languages, please feel free to do so!
May 04, 2015

I made a little rubric for evaluating the design of Microservices. Keep in mind that it is still very preliminary. If you have comments, I would be more than happy to discuss them with you.

Find it here:

https://github.com/microservices-course/material/blob/master/rubrics/design.md
March 17, 2015
Posted by Claudio Guidi
Wordpress developers could have the problem to integrate the website they are developing with some extra data which come from external resources. Data synchronization could be not easy, it must be scheduled, data are visible on the web application with a period equal to the synchronization timeout and it usually requires a level of maintenance that is high with respect to the amount of synchronized data.

In these cases it could be useful to synchronously call an external server in order to retrieve the data we need. Jolie could be very helpful for developing a fast API server to be integrated within your web site.

This short post describes a way for easily integrate Wordpress with a Jolie service.

First of all, let me consider the simple case of getting a list of addresses from a Jolie service. This service could be written as it follows:

type GetAddressRequest: void {
    .name: string
    .surname: string
}

type GetAddressResponse: void {
    .address: string
}

interface ServerInterface {
    RequestResponse:
        getAddress( GetAddressRequest )( GetAddressResponse )
}


execution{ concurrent }

inputPort AddressService {
    Location: "socket://localhost:9001"
    Protocol: http
    Interfaces: ServerInterface
}

init {
    with( person[ 0 ] ) {
        .name = "Walter";
        .surname = "White";
        .address = "Piermont Dr NE, Albuquerque (USA)"
    };
    with( person[ 1 ] ) {
        .name = "Homer";
        .surname = "Simpsons";
        .address = "Street 69, Springfield USA"
    };
    with( person[ 2 ] ) {
        .name = "Sherlock";
        .surname = "Holmes";
        .address = "221B Baker Street"
    }


}

main {
    getAddress( request )( response ) {      
        index = 0;
        while( index < #person
                && person[ index ].name != request.name
                && person[ index ].surname != request.domain ) {
            index ++
        };
        if ( index == #person ) {
            throw( PersonNotFound )
        } else {
            response.address = person[ index ].address
        }
    }

}


In this case I simulated a database with a simple vector of persons. The service is located in socket://localhost:9001 and you can easily invoke him through a browser by using this url: http://localhost:9001/getAddress?name=Sherlock&surname=Holmes

Clearly, this is an example which runs on your local machine, it is simple to export it in a distributed scenario by simply running Jolie on the remote machine, let me suppose it is located at IP X.X.X.X.

When the Jolie service is executed, we can invoke it within a wordpress page. In order to do this, simply add the following PHP code on the page where you want to show the results:

$data = array(
    'method' => 'POST',
    'timeout' => 45,
    'httpversion' => '1.0',
    'blocking' => true,
    'headers' => array(),
    'body' => array( 'name' => 'Sherlock', 'surname' => 'Holmes' )
    );

$url = 'http://X.X.X.X:9001/getAddress';
$result = wp_remote_post( $url, $data );

print_r( $result );

if( is_wp_error( $result ) ) {
echo $result->get_error_message();
}


Where $result contains the response message.

You can also exploit a JSON format message by adding the following parameter to the http protocol of the Jolie service and re-arranging the PHP code for managing JSON messages.

 inputPort AddressService {
    Location: "socket://localhost:9001"
    Protocol: http { .format="json" }
    Interfaces: ServerInterface
}


Moreover you could connect the Jolie service to a database for retrieving the data you need, check the Jolie tutorial page for getting how to use databases in Jolie.



February 06, 2015
In Microservice architectures, software components are built as reusable and scalable services that interact with each other. While this introduces many advantages, it also brings all the issues of distributed computing to the internals of software applications and developers need to be careful about this. Luckily, we are standing on the shoulders of all the knowledge that we accumulated by using Service-oriented Architectures (SOAs), so we are more aware of these issues and we have more technologies and methodologies to deal with them. Microservices could (and I believe they will) fly after all, in one form or the other, and we at the Jolie team are surely not the only ones sharing this optimism.

But like SOAs were, Microservices are also at risk of exposing programmers to too much complexity to deal with when they take their first steps in this world. And we are at risk again of making arbitrary technology choices that we may regret later, just like SOAP was in many cases.

In developing Jolie, a microservice-oriented programming language, we are trying to make microservice programming simple and productive. A major challenge in this is how we can map design ideas for a microservice architecture to code as fast as possible, without committing to specific implementation details on communications and service deployment/distribution.

The problem of focusing on design issues and not getting locked inside of interoperability problems is particularly dear to me. Jolie helps in this, e.g., by abstracting application logic from the underlying data formats that you will use for communications.

Another problem that is dear to the team in general and I will talk about below is that of making composition of microservices easy. This does not simply mean composition of the operations offered by microservices, but also dealing with how we can deploy new systems based on previously developed systems.

But talk is cheap. Let's do code and see an example.


Get Jolie

You don't need to have Jolie installed to read this article, but if you want to try what I will show, go install Jolie.
Ready? Let's proceed.


A calculator microservice

We are going to program a little toy calculator microservice for making additions. Feeling lazy? Download the code here.
Open a file and call it calculator.ol.
Then we write:

include "calculator.iol"

execution { concurrent }

inputPort CalcInput {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: CalcIface
}

main
{
sum( req )( resp ) {
resp = req.x + req.y
}

}

Above, we are telling Jolie that this microservice is concurrent, i.e., each request is handled in parallel by a lightweight process (Jolie processes are implemented as threads with a local state).
The input port CalcInput states that the service can be accessed using TCP/IP sockets on port 8000, using the SODEP protocol. CalcIface is the exposed interface, which we define in  the included file calculator.iol:

type SumT:void {
.x:int
.y:int
}

interface CalcIface {
RequestResponse: sum( SumT )( int )

}

CalcIface is an interface with only one Request-Response operation called sum, which receives a tree with two nodes (x and y, both integers) and returns an integer.
The sum operation is implemented in the main procedure of the calculator microservice: it simply receives the message on variable req and replies with the sum of the two subnodes x and y to the invoker.


Using the calculator

Using the calculator service from Jolie is easy. We create a client.ol program, include the interface and off we go:

include "calculator.iol"
include "console.iol"

outputPort Calculator {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: CalcIface
}

main
{
with( req ) {
.x = 3;
.y = 2
};
sum@Calculator( req )( resp );
println@Console( resp )()
}

Running this client program will reach the calculator service and print the number 5 on screen. SODEP is a binary protocol, so you don't need to worry about overhead as much as you have to in, e.g., SOAP.

The attentive reader may have noticed that in our client code we are calling the println operation of the Console service, just like we are calling the sum operation of calculator. That's right, even our Console library is a microservice that you can relocate as you like.

OK, but what if the rest of my system is not in Jolie?

Jolie is interoperable with many other technologies, both server- and client-side. Let's see what happens if our client is not written in Jolie and does not support SODEP. Let's say instead that it's a web browser and that it consequently uses HTTP.
Take the calculator program again and add the following before the main procedure:

inputPort CalcWebInput {
Location: "socket://localhost:8080"
Protocol: http
Interfaces: CalcIface
}

Et voilà, now calculator is not only exposed on TCP port 8000 with SODEP but also on TCP port 8080 with the HTTP protocol. Run calculator and you can now point your browser to:

http://localhost:8080/sum?x=3&y=5

You will see calculator crunching the numbers and reply with:

<sumResponse>8</sumResponse>

You can use many other protocols (HTTP/JSON, SOAP, local memory, ...) and communication mediums (Bluetooth, UNIX sockets, ...), depending on your needs. See here.


What if I cannot change the code of my microservices?

If your calculator is a black box that you cannot or do not want to touch, then you cannot just edit its code to add an HTTP input port. This is the typical case in which you would like to compositionally evolve your system without touching what has already been deployed.

The standard solution is to use a proxy. In our example, we wish to make a proxy that enables web browsers to communicate with the calculator service:

Web browser <-> Proxy <-> Calculator

Proxies (and variants) are very useful in general as intermediate glue components in service systems, but managing them may require you getting to know yet another technology. In Jolie, instead, proxies are powered by a native language primitive for forwarding messages and integrate well with our communication ports, so you can take advantage of the data abstraction layer provided by Jolie.

Open a file proxy.ol and write:

include "calculator.iol"
include "console.iol"

execution { concurrent }

outputPort Calculator {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: CalcIface
}

inputPort ProxyInput {
Location: "socket://localhost:9000"
Protocol: http
Aggregates: Calculator
}

main
{
in()
}

The important part here is that we have an input port on TCP port 9000 that aggregates the output port towards the Calculator. This means that all messages on port ProxyInput for an operation provided by Calculator will be forwarded to that service. So now we have our proxy and we can navigate to:

http://localhost:9000/sum?x=3&y=5

The proxy service will now receive your HTTP message on port ProxyInput, see that it is for the sum operation of calculator. The message is therefore converted to the SODEP format, as specified by the output port towards the calculator. The response from calculator will finally be converted again from SODEP to HTTP and sent to the initial client.

Aggregation is really useful. You can also add hooks (called couriers) for intercepting messages and, e.g, log operations or check for authentication credentials.

Workflows

Alright, now we have our toy microservice system. Let's say that after a while, you want to add the requirement that a user must log in before calling the sum operation on the calculator. This is a workflow, the magic keyword being before in the previous sentence. Jolie inherits native workflow primitives from business process languages (e.g., BPEL), making handling flows of communications among services a breeze!

Open calculator.ol and change the main definition as follows:

cset {
sid : SumT.sid
}

main
{
login( creds )( csets.sid ) {
if ( creds.pwd != "jolie" ) {
throw( InvalidPwd, "The password is jolie" )
};
csets.sid = new
};
sum( req )( resp ) {
resp = req.x + req.y
}
}

The cset block above declares a session identifier sid, which we will use to track invocations coming from a same client (that's a correlation set). We also introduced a new operation, login, which simply checks if the password sent by the client is jolie and otherwise throws a fault to the invoker. Observe now that we use the semicolon operator to tell Jolie that sum becomes available only after login has successfully completed.

This is a fundamental difference with respect to, say, object-oriented computing, where if you want to enforce order in how different methods are called, you have to implement it yourself with bookkeeping variables and concurrent lock mechanisms. Here instead, you just write a semicolon.

We also need to update the interface of the calculator, in calculator.iol, to account for our modifications:

type SumT:void {
.x:int
.y:int
.sid:string
}

type LoginT:void {
.pwd:string
}

interface CalcIface {
RequestResponse:
login( LoginT )( string ) throws InvalidPwd( string ),
sum( SumT )( int )
}

We are done! No need to update the code of the proxy, as the aggregation primitive is parametric on the interface and the protocols used by the aggregated services. Jolie will do that lifting for us. So now we can browse to: 

http://localhost:9000/login?pwd=jolie

You will get a random session identifier. I got:

<loginResponse>193e6a7a-895c-4ff6-b32a-5bbab0eea7f3</loginResponse>

So now I can go to (remember to replace sid with what you got in the previous call if you try this yourself):

http://localhost:9000/sum?x=3&y=5&sid=193e6a7a-895c-4ff6-b32a-5bbab0eea7f3

And receive my good old 8 as result.

Have a look at the Jolie documentation if you are interested in using cookies and theming your responses with nice HTML. You can put that in a separate microservice, to keep calculator clean!

Conclusions

This is just the tip of the iceberg of what you may have to do with microservices and what Jolie can do for you.

But we can already see some of the underlying concepts that Jolie is based on:

  • You should not commit to any particular communication technology. Microservices are born to be lean and reusable, and should be kept decoupled from the implementation details of how data is exchanged.
  • Building proxies and other kinds of architectural composers is very handy in microservice architectures just like it is in any other distributed system. However, with microservices you can end up having a lot of them. Jolie helps in creating and managing these composers by making them programmable with native primitives that make it clear what is happening.
  • Workflows can help in making the structure of communications followed by a service explicit. They should be easy to write and Jolie strives in doing so. See here for our other workflow primitives.


Stay tuned for the next chapters. :-)

Meanwhile, you may want to have a look at some more material here and here.
February 03, 2015
Posted by Claudio Guidi
Recursion in service oriented architectures is something unusual because it seems to be useless. So, why a post about recursion in SOA? I think the big result we can obtain from service oriented recursion is understanding service oriented programming paradigm nature and technologies. Recursion indeed, is a well known programming pattern hugely adopted by all the programmers in the world and its usage could reveal us some interesting features about a programming language. So, I want to use recursion in a SOA because I want to know more about SOA. Let's do it.

In the following example you can find the implementation of Fibonacci with Jolie.
[ You can copy it into a file and save it with name fibonacci.ol. Then run it typing the command
jolie fibonacci.ol ]


include "console.iol"
include "runtime.iol"

execution{ concurrent }

interface FibonacciInterface {
  RequestResponse:
    fibonacci( int )( int )
}

outputPort MySelf {
  Location: "socket://localhost:8000"
  Protocol: sodep
  Interfaces: FibonacciInterface
}


inputPort FibonacciService {
  Location: "socket://localhost:8000"
  Protocol: sodep
  Interfaces: FibonacciInterface
}

main
{
  fibonacci( n )( response ) {
    if ( n < 2 ) {
      response = n
    } else {
      {
fibonacci@MySelf( n - 1 )( resp1 )
|
fibonacci@MySelf( n - 2 )( resp2 )
      };
      response = resp1 + resp2
    }
  }
}




The code is very intuitive and simple. The outputPort MySelf defines the external endpoint to be invoked which corresponds to the input endpoint FibonacciService where the operation fibonacci is deployed. The invocations are performed in parallel (operator |) and they are blocking activities which wait until they receive a response from the invoked service
. You can invoke it by simply exploiting the following client which sends 10 as input parameter:

include "console.iol"

interface me {
  RequestResponse:
    fibonacci( int )( int )
}

outputPort Service {
  Location: "socket://localhost:8000"
  Protocol: sodep
  Interfaces: me
}

main
{
  fibonacci@Service( 10 )( result );
  println@Console( result )()
}

Starting from this simple example we can understand some interesting features:

server sessions represent recursive call stack layers: each invocation opens a new session on the server which represents a new layer in the recursive call stack.

loosely coupling: each invocation is separated from the others which guarantees that the value of n is not overwritten by different calls.

runtime outputPort binding: the binding of the output endpoint (MySelf)  must be achieved at runtime and not at deploy time in order to avoid frustrating programming issues like those reported here:  http://blogs.bpel-people.com/2007/02/writing-recursive-bpel-process.html.

the invocation stack could be distributed: you can imagine to deploy more than one fibonacci service and switch the invocations from one to another depending on some inner parameter such as, for example, the number of the open sessions. As an example consider the code modified as it follows:

init {
  getLocalLocation@Runtime()( global.location );
  MySelf.location = global.location;
  println@Console( MySelf.location )();
  global.count = 0
}

main
{
  fibonacci( n )( response ) {
    synchronized( lock ) {
      global.count ++;
      println@Console( "begin n="+n )();
      if ( global.count >= 100 ) {
MySelf.location  = "socket://localhost:8001"
      }
    };
    if ( n < 2 ) {
      response = n
    } else {
      {
fibonacci@MySelf( n - 1 )( resp1 ) 

fibonacci@MySelf( n - 2 )( resp2 )
      };
      response = resp1 + resp2;
      synchronized( lock ) {
global.count --;
println@Console( "end n="+n )();
if ( global.count < 100 ) {
 MySelf.location  = global.location
}
      }
    }
  }
}

Here the location of the outputPort MySelf can be changed dynamically during the execution. global.count stored the number of the current open sessions, if it is greater than 100 the location is changed into socket://localhost:8001 where the second fibonacci service is deployed. In this way you can easily create a chain of Fibonacci services whose sessions participate to a unique Fibonacci number recursive calculation.


Conclusions
Here I used service oriented recursion for programming a Fibonacci service with Jolie which can be invoked by external clients. This service could be also chained with other copies of it in order to obtain a distributed SOA for calulating the Fibonacci number recursively. Such an example could be an interesting reference point for understanding how service creation and invocations work in a SOA.


January 30, 2015
Lately I have been asked a lot how to run a Jolie program from inside of a Java program.
The opposite, running Java from Jolie, is well documented in our documentation site.

Running Java code from inside a Jolie program is also a very simple process.
First, you need to add to your classpath (import in your project, if you are using an IDE) the jolie.jar and libjolie.jar libraries that you can find in your Jolie installation directory (JOLIE_HOME).
Now, from Java, you can create an Interpreter object. Interpreter is the class that implements the Jolie language interpreter. It will need two parameters for the constructor. The first is the command line arguments that you want to pass to the interpreter. The second is the parent class loader to use for loading the resources that the interpreter will need. Here is an example based on the Linux launcher script (but it should work on any OS), where we assume that the Jolie program that we want to run is called main.ol:


String jh = System.getenv( "JOLIE_HOME" );
String args[] = "-l ./lib/*:$JOLIE_HOME/lib:$JOLIE_HOME/javaServices/*:$JOLIE_HOME/extensions/* -i $JOLIE_HOME/include main.ol".replaceAll( "\\$JOLIE_HOME", jh ).split( " " );
final Interpreter interpreter = new Interpreter( args, this.class.getClassLoader() );

Now you just need to use the run method of the interpreter object you have just created. Here I do it in a separate thread, as that method will return only when the Jolie program terminates. This is a toy example so I am disregarding exceptions here, but you should obviously care about them in real-world code:

Thread t = new Thread( () -> {
try { interpreter.run(); }
catch( Exception e ) { /* Handle e.. */ }
} );
t.setContextClassLoader( interpreter.getClassLoader() );
t.start();

If you need to stop the interpreter manually, you can use the exit method it comes with. It takes a hard timeout if you need one. Enjoy your Jolie interpreter inside of Java!
January 26, 2015
Thanks to Matthias Dieter Wallnöfer, Jolie now supports the negotiation of HTTP compression and this is activated by default. This means that all your Jolie-powered websites will benefit from this, along with your Jolie programs which access remote HTTP content.

We have just uploaded the patch to the web server on the Jolie website: http://www.jolie-lang.org

We have also added caching, with the good old .cacheControl HTTP configuration parameter that I introduced a while ago (http://arxiv.org/abs/1410.3712).

Two minutes later, the loading speed of the main webpage doubled. We went from ~140ms down to ~70ms ! You should feel the difference navigating from one page to the other (aside from those pages using slideshare plugins, which take time to load from slideshare.net).

Enjoy a faster Jolie website and faster $all_your_Jolie_programs.

PS: Yes, this will be released with Jolie 1.1.1
December 19, 2014
The first university course almost entirely on Jolie (and choreographies!) ended today, with cakes and final discussion on the exam procedure. :-)
I had a lot of fun teaching it. I look forward to the next edition, I got lots of good ideas from this one.

It was also a lot of work, as I had to prepare a lot of material from scratch. There is still lots of room for improvement, but I got a decent set of lectures and exercises out of it. Here is the material from this year:
http://fabriziomontesi.com/teaching/imds-f2014/index.html

Feel free to use it, as long as you put a reference to the origin please. Or, just contact me.

This year I also experimented with teaching students how to prepare and give a presentation on a scientific paper, and how to read scientific articles. I think it really paid off, they gave excellent presentations. But it's hard to tell, I had really clever students.. so maybe it's all them. ;-)
Here are some of their carefully crafted presentations:
I still have to get the final feedback on the course, but an initial probing tells me that most of them would use Jolie for their future software projects. This alone makes all the effort worth it. I can't wait to see what they build with it. Some of them have already started.

As the students were so happy, I am going to try carrying out an educational project on research-led teaching next year and see how to push its limits a little more every year! Doing live demos on Jolie during lectures is so much fun (you should see the faces of people when I hack a multiparty web chat server or a prototype cloud computing solution in Jolie in about 10 minutes), but I think that I can transform some of them into videos and apply some more flipped classroom techniques along with other videos from the research community on session programming. We'll see.. and suggestions are welcome!

PS: something that baffled me. I could not find a good tutorial for master students on CCS and the pi-calculus. I had to prepare introductory lectures by myself, which was also quite a bit of work. But maybe they are out there and I just could not find them?
August 21, 2014
Posted by Claudio Guidi
In this post I would like to discuss how a web server should be considered in the context of a Service Oriented Architecture or a distributed system in general. Usually, the web server is used as a tool which enables the possibility to public files and applications  under the HTTP protocol. Indeed, applications are developed by using some specific technology like PHP, Ruby, Java, etc. and then deployed into a web server for making them available on the web. The same approach is adopted also in Service Oriented Architecture where web services and orchestrators are developed in a given technology and then published in a web server together with the WSDL documents.

Here I want to share one of the results we obtained by developing Jolie as service oriented programming language. In Jolie the web server is just a service which provides its operations under an HTTP protocol and it follows the same programming rules we use for simple services and orchestrators. Here you can find the code of Leonardo, which is a web server completely developed in Jolie:

http://sourceforge.net/p/leonardo/code/HEAD/tree/trunk/leonardo.ol

You can download the code here and try to execute it by simply typing the following command in a shell:

      jolie leonardo.ol www/

where  www/ is the root folder where files are retrieved by Leonardo (remember to create it if it is missing). Now try to add a simple index.html file into the www folder like the following one:

<html>
<head/>
<body>
Hello World!
</body>
</html>

Then open your browser and set the url: http://localhost:8000 and you'll see the html page displayed in it. Try also to add images, css, html pages and subfolders into the www/ folder.
You can change the port by setting the Location_Leonardo parameter of the config.iol file. 

But,
          how can I program the server side behavior of a web application?

And here comes the service oriented approach. First of all let me introduce a new operation called test into the HttpInput by modifying its interface as it follows:

interface HTTPInterface {
RequestResponse:
default(DefaultOperationHttpRequest)(undefined),
test
}

Then let me add the implementation of the test operation into the main:

main {
[ default( request )( response ) {
/* ... code of the default operation */
} ] { nullProcess }

[ test( request )( response ) {
format = "html";
response = "Test!"
}] { nullProcess }
}

Operation test executes a very simple piece of code that is:

  • setting the response format to "html" in order to make the browser able to manage the response as an html file
  • set the response message with a simple html page

Now if we relaunch Leonardo and we point the browser to  http://localhost:8000/test we will see the page generated by the test operation. 

This is a pretty standard way for programming a web application: joining a web page to each operation. You can do it but I don't like it, even if in some cases it could be useful. I prefer to completely decouple the server side from the client side. I prefer to public the operation test as a JSON API which can be used by the client using an AJAX call. In this way I can program the server side as a service and the client side as a dynamic application. In order to this I modify the interface for introducing the message types which are very useful for catching message faults:
type TestType: void {
.message: string
}

interface HTTPInterface {
RequestResponse:
default(DefaultOperationHttpRequest)(undefined),
test( TestType )( TestType )
}

Then I modify the implementation of the test operation:

[ test( request )( response ) {
response.message = request.message + " RECEIVED!"
}] { nullProcess }

Finally, I just insert a little bit of Javascript on the client side by modifying the index.html:

<html><head>
  <script type="text/javascript" src="http://code.jquery.com/jquery-2.0.3.min.js"></script>

  <script>
 function jolieCall( request, callback ) {
     $.ajax({
          url: '/test',
          dataType: 'json',
          data: JSON.stringify( request ),
          type: 'POST',
          contentType: 'application/json',
          success: function( data ){callback( data )}
     });
 }
 
 function afterResponse( data ) {
$( "#test" ).html( data.message )
 }
  </script>
  
</head>
<body>
  <div id="test">
<button onclick="jolieCall( {'message':'Ciao'}, afterResponse )">TEST</button>
  </div>
</body>
</html>

That's all! Try to open the index.html page on the browser and click on TEST button. It is worth noting that Jolie recognize the JSON format and sends back a JSON message.

Now, try to add other operations and create a more complex web application.
Have fun!


August 21, 2014
Posted by Claudio Guidi
Some months ago I read this article about microservices and I found it very interesting because of the experience we have had so far in the development of Jolie language. Jolie was born as a language for crystalizing SOA principles in a specific domain language but, day by day, we were becoming more and more confident that the boundaries of SOA were not enough for dealing with some issues that we had encountered during programming with Jolie. Thus, I was very surprised and excited when I read about microservices because I think they enlarge the domain  of service oriented programming in a way which is coherent with the results we have had so far. This is why I would like to share our experience with the microservices community. On the one hand my aim is to promote the usage of Jolie as one of the technologies which could help in the design and the development of microservices, and on the other hand I hope to give my contribution in the definition of concepts and practices for microservices. In particular, in this post, I would like to introduce a simple concept called Service Threshold which could be of help when reasoning about service and microservice componentization. The main idea behind the service threshold is to provide an abstract boundary which allows for the separation among a world of services and a world of components, where a component is a general piece of software.

The Service Threshold
The Service Threshold is the architectural line which separates services from other components (that are not services) as shown In Fig 1. where I simply represent services as hexagons and other components as squares.

Fig 1. The Service Threshold abstractly separates services (hexagons) from 
components that are not services (squares).

It is a very simple concept but some questions need to be addressed:

Why do I need to introduce it?
A line always represents a division, in this case a logical division between two kinds of things that we consider different. I introduce it because it helps me to change my perspective about distributed software architectures. Indeed, I need to think of them in a way which allows me to design, deploy and manage them quickly and easily. Summarizing, I need that line because I am looking for a simplification of distributed and heterogeneous systems engineering. 

Are there different properties above and under the service threshold?
Clearly, there are differences between the top plane and the bottom one. If I operate "above" the line I would like to be in a world where a service is the basic element which can be composed and managed by following some specific rules related to services, no other kind of software components exist. Otherwise, if I am "under" the line, the service is just something I have to create, it is the target of my work.

Which are the differences between a service and component?
A service is an autonomous running piece of software able to provide its functions when invoked. Its execution does not depend on components out of those strictly required.  The service functions should be always available for invocations, if not the service should reply with a specific fault message. Invocations are always achieved by means of message exchanges which could be synchronous or asynchronous. It does not matter the transport protocol used for the message passing. A service can be stateless or stateful.
A component is a piece of software which cannot be considered as a service.

Which is the right place to put the threshold line?
This is the core question every distributed software architect should deal with. Depending on the answer, I would design a different system with different properties. If I put this line very high I will have few services and a lot of components, if I put this line at a low level I will have a lot of services and few components. So, the question could be changed in: how many services do I need? Again, which are the basic properties that I need to take into account in order to identify all the required services?

The service threshold is not a line but a boundary
A distributed system is a system where software artifacts are separated, thus even though I have discussed the service threshold as a horizontal line, it fits better to think about it as a boundary which allows me to isolate and identify the different services involved into a system.


Fig 2. The Service Threshold is a boundary which allows me to isolate and identify the services.

Once defined, the service  threshold allows me to focus on the single service without being worried about its connections too much. I can just limit my effort to the development of the service as an autonomous specific provider of functions. The final target is to provide a basic set of services which represent the core functions of the application I am interested in. Some of them could be bought from a third party supplier, the others could be developed from scratch. The most important think is that each service function must be well separated from the others and, at the same time, it must be strictly necessary to the overall system. As an example, let me consider the human body where each organ supplies a well defined function which is fundamental for life. They are precisely separated by means of different tissues and they are characterized by different biological properties. 

All of them are needed, because the functions that they supply are needed, nothing is too much, nothing is too less. They are connected in a perfect and inexpensive way, because they all participate to form the human body, a more complex and completely different organism. 

Service connections and dependencies
Once all the services have been identified, I can deal with service communication and connections. A communication is performed when a message is sent by a service and received by another one, where a message is a limited set of data transmitted in a limited interval of time. A connection represents the possibility to perform a communication between two services: a sender and a receiver. Usually a service receiver is not aware of the sender (this is not true in the case of a stateful long running transaction). A connection becomes a dependency when the sender needs to invoke another service in order to accomplish its own tasks.
Fig 3. Service A has a dependency on service B.
Service dependencies allow me to divide services in two main categories: 
  • Autonomous services
  • Depending services
Autonomous services usually control resources such as databases, electronic devices, computational resources, legacy systems, etc. These kind of services are the software artifacts which enable the integration of  the underlying resource with a system of services and the APIs that they provide represent the only way for interacting with it. They are usually designed to be independent from the context they will be inserted in and they are strongly loosely coupled.

Depending services are built upon other services because they require to call other services to accomplish their tasks. They could be:
  • Mixed services
  • Pure coordination services
Mixed services control resources as the autonomous ones (e.g. a database) but they also need other information from other services for providing all their functions.
Pure coordination services do not control any data resource but they just call other services for achieving their tasks (orchestrators are usually pure coordination services).
 

Fig.4 Autonomous services, Mixed services and Pure coordination services
By componing these kind of services it is possible to obtain a complex distributed system which provides a new set of functions different from those of the constituent ones. I call service complex system a distributed system whose components are all connected services.

Fig. 5  A service complex system is a distributed system whose components are all connected services.


The service threshold can be applied over a set of services
A service complex system could be formed by tens or hundreds of services. Clearly, when the number of services is very high it could be difficult to manage and maintain the system because of its complexity. In this case, in order to simplify the management, I exploit another interesting property of the service threshold: I apply it over a set of existing services by grouping them. The services grouped by a service threshold can be seen and treated as a single service. In this case all the inner connections are hidden for an external observer and the only ones that are relevant for us are those that can be observed outside the service threshold boundary. 
Fig 5. The service threshold can be applied over a set of services in order to group them
and identify them as a single service.
There are no limits to the application of the service threshold over a set of services, thus I could apply it again and again in order to reduce the system to a limited set of services. Since a service is usually stateless and always ready to serve different clients concurrently, it is possible that the same service could be enclosed by two (or more) different service thresholds as in the following picture:

Fig 6. A service could be enclosed by two different service thresholds. 

In this case the resulting services both contain the same service inside. Is this possible? Yes, in the following section I will answer to this question.

Primary thresholds and abstract thresholds
As I said before a service is always a running piece of code. Thus, a set of files in our file system is not a service, it is just a set of file which could contain executable code but they are not a service. They start to become a service when they run. This is a fundamental point because a service exists only if it is able to respond to our requests. This is why it is possible to double a service by wrapping it into two different service thresholds as represented in Fig 6. It is possible because I am grouping the service functions, not the piece of code. I call abstract service threshold all the service thresholds which group existing services. When I apply the abstract service threshold to a set of services I loose the deployment details of a service (where it is, how it is composed, which technology it uses, etc) because I am focusing only on its functions. On the contrary, the primary service threshold is applied to a set of components that, when executed, they become a service. By definition. the primary service threshold has a completely different nature with respect to the abstract one because it will be applied to components for tting services. Nevertheless, I both call them service threshold because they both allow me to identify a new service even if they are applied to different domains.

Fig 7. The Abstract Service Threshold (on the left) groups existing services, the primary service threshold groups a set of components that, when executed, they become a service.

The primary threshold is a technological threshold
The Primary Threshold is a technological threshold because it clearly defines the boundary between service composition mechanisms and other technologies mechanisms. It does not matter which kind of technology I select, but it is important that I will be able to satisfy some minimum requirements that allow me to create a service (I suppose you would ask me which are the minimum requirements for creating a service, wait wait wait I will discuss them in another post). Within the primary threshold I could exploit any kind of technology and any kind of communication approach (such as in memory data exchanges, file exchange, etc.).  I can also use specific applications such as web servers, application servers, etc. The final target is to prepare the basic software layer which allow me to jump into the service world. Everything happens inside this threshold is a matter strictly related to the technologies I chose for building the service. This point is important because allow me to state that there exists a trade off between the benefits obtained by the introduction of a service and the technological overhead I have to pay. Is it simple to maintain a service developed with a specific technology? Which skills I require for achieving the development? Does the technology easily scale with the service load? Moreover, do I need to adopt the same technology for all the services of our system or do I need to change it? When? Will I be able to maintain the required skills for managing all the existing services I have? These are not trivial issues because more is the number of technologies I adopt, more is the human resource knowledge of my team I have to manage. Indeed, if I adopt a technology I need someone who is able to manage it. 

Conclusions
In this post I discussed service and microservices componentization by introducing the concept of service threshold which is a useful tool for approaching distributed systems based on services and microservices. I hope this post could be useful for those people that are trying to approaching service system design in a simple and intuitive way. Service thresholds could be a useful means for structuring the design and defining the development phases of a service system.  


May 13, 2010
No, this is not related to *buntu. But this is looong... and now I've got your attention. :-)


Interacting with Web Services usually implies reading some WSDL (Web Service Description Language) document. Unfortunately, WSDL documents are written in XML and can be pretty complicated, leaving the user with no other choice than to use some graphical development tool in order to understand them.

Lately at italianaSoftware we had to handle a lot of Web Services stuff, equipped with long WSDL documents and complex data types. Jolie already had support for the SOAP protocol, but one had to code the interface and data types of the Web Service to invoke by hand. So we started a new project aiming to enable Jolie for the automatic usage of WSDL documents, mainly composed by two parts: a tool and an improvement to the Jolie SOAP protocol. Of course both things are open source and available in the trunk branch in the Jolie svn repository!

wsdl2jolie (whose executable is now installed by default if you install Jolie from trunk) is the newly developed tool in question. It simply takes a URL to a WSDL document and automatically downloads all the related files (e.g., referred XML schemas), parses them and outputs the corresponding Jolie port/interface/data type definitions.
Let us see an example. See this WSDL document for a service that calculates prime numbers: http://www50.brinkster.com/vbfacileinpt/np.asmx?wsdl. Reading the raw XML is not so easy, or at least requires some time.
Let us see what we get if we execute wsdl2jolie http://www50.brinkster.com/vbfacileinpt/np.asmx?wsdl :

Retrieving document at 'http://www50.brinkster.com/vbfacileinpt/np.asmx?wsdl'.
type GetPrimeNumbersResponse:void {
.GetPrimeNumbersResult?:string
}

type GetPrimeNumbers:void {
.max:int
}

interface PrimeNumbersHttpPost {
RequestResponse:
GetPrimeNumbers(string)(string)
}

interface PrimeNumbersHttpGet {
RequestResponse:
GetPrimeNumbers(string)(string)
}

interface PrimeNumbersSoap {
RequestResponse:
GetPrimeNumbers(GetPrimeNumbers)(GetPrimeNumbersResponse)
}

outputPort PrimeNumbersHttpPost {
Location: "socket://www50.brinkster.com:80/vbfacileinpt/np.asmx"
Protocol: http
Interfaces: PrimeNumbersHttpPost
}

outputPort PrimeNumbersHttpGet {
Location: "socket://www50.brinkster.com:80/vbfacileinpt/np.asmx"
Protocol: http
Interfaces: PrimeNumbersHttpGet
}

outputPort PrimeNumbersSoap {
Location: "socket://www50.brinkster.com:80/vbfacileinpt/np.asmx"
Protocol: soap {
.wsdl = "http://www50.brinkster.com/vbfacileinpt/np.asmx?wsdl";
.wsdl.port = "PrimeNumbersSoap"
}
Interfaces: PrimeNumbersSoap
}

which is the Jolie equivalent of the WSDL document. Those ".wsdl" and ".wsdl.port" parameters are the aforementioned improvement to the SOAP protocol: when the output port is used for the first time, Jolie will read the WSDL document for processing information about the correct configuration for interacting with the service instead of forcing the user to insert it by hand.
Now we can just put this into a file, say "PrimeNumbers.iol", and use the output ports we discovered from Jolie code. As in the following:

include "PrimeNumbers.iol"
include "console.iol"

main
{
request.max = 27;
GetPrimeNumbers@PrimeNumbersSoap( request )( response );
println@Console( response.GetPrimeNumbersResult )()
}

That little program will output "1,3,5,7,11,13,17,19,23". The example can be downloaded from this link (remember, it requires Jolie from trunk).

wsdl2jolie acts as a nice tool for being able to use the typed interface of a Web Service from Jolie and for getting a more human-readable form of a WSDL document (its Jolie form).

Another nice feature we get for free from the SOAP protocol improvement is that now MetaService can act as a transparent bridge towards Web Services. Simply call the addRedirection operation with the right protocol configuration (in this case, the ".wsdl" and ".wsdl.port" parameters) and MetaService will automatically download the WSDL document (we also have a cache system for them already in trunk!) and make it callable by clients.
This is a huge step forward for client libraries such as QtJolie: just tell where the WSDL document is and you can already place calls to the Web Service of interest.
Also, by using wsdl2jolie first and then tools such as jolie2plasma one could use the Jolie intermediate representation for transforming a Web Service interface definition into a (KDE) Plasma::Service XML one. With the same trick, one could also write C++ generators for QtJolie and introduce ease and type-safeness to Web Services invocations. I and Kévin sure have things to think about now. ;-)

Beware that the code is still in its early stages. It already works with a lot of Web Services, but we're still improving it. For instance, we currently support only the SOAP document format (which is the one people advise to use, anyway). Patches, comments and reports are welcome in jolie-devel @ lists.sourceforge.net!
September 09, 2009
Jolie is getting into shape for the 1.0 release. This is requiring a lot of work, and whenever we reach our current objectives we feel like improving things even further (every developer likes perfection, the problem is it requires too much time)! Nevertheless, the line for a good 1.0 release has been placed and is getting very near. The two major points we need to address are SSL support (both in SODEP an HTTP) and documentation. By documentation I'm not just referring to the language documentation but also to the source code documentation of the interpreter; the aim is to make the Jolie interpreter more approachable by hackers who want to improve it or want to write an extension (protocols, specifications, hardware support, ...). The 1.0 release will also contain experimental support for XML-RPC.

SSL support is already working and waiting for inclusion (rough edges need to be investigated) in the main source tree. SSL support will particularly benefit B2B applications, which until
now had to get security by wrapping Jolie in SSL containers, and the KDE project, which has just included support for remote services in trunk using Jolie and MetaService as backend technologies. SSL support in Jolie would allow KDE remote Plasma services to encrypt their data exchanges, an essential feature when dealing with sensitive data (though the initial handshake part is already made secure by using Qt).
I must say that the KDE inclusion was pretty quick: it was just one year ago when the Jolie and the Plasma teams met at the first Tokamak and now we already have a first functioning version in trunk. This process required quite a lot of work and skill, from writing a service orchestrator that could dynamically create bridges from Plasma to services using different protocols and technologies onward to the QtJolie gluing layer and the final implementation in the Plasma library. Now we have the basis for constructing even more complex frameworks and applications, and all of this will benefit from the future Jolie static analysis tools that we (the Jolie team) are planning to implement. All of this to say that it really is a pleasure to work with the KDE team: I will be sure to keep in touch as much as possible.

I'm now focusing heavily on source code documentation and polishing (for which I've just made some pretty big commits). Regarding code polishing, I got some help from the FindBugs static analysis tool; you can just ask it to read some Java bytecode and it comes up with a lot of good hints for performance improvements and bug removals. It is really easy to use and it even considers correct resource locking and releasing. Pretty useful if you have old code to maintain or for checking new extensions before merging them.


Ah yes, I should also blog about a lot of other features and the porting of Jolie to ARM-based devices thanks to the Jalimo project... but that's for other blog posts!
January 21, 2009
Today I have moved MetaService from JOLIE's playground source tree to the main source tree, the one that gets installed when you type ant install.

A short description for people that do not know what MetaService is: MetaService is a JOLIE service that allows you to dynamically bridge applications that speak different protocols (sodep, soap, http, etc.) or use different communication transports/mechanisms (sockets, local sockets, Java RMI, etc.). It has been developed as an answer to the Plasma project's needs to have a transparent means to introduce Plasma to the Service-oriented world. The project turned out to be so useful that I have built a Java library for interacting with and dynamically loading it (metaservice-java, to be found in trunk/support/metaservice-java) which we, as italianaSoftware, have already used in the scope of web application development (I can not say much more on this topic... yet ;).

As a bonus, *nix users get a launcher script for MetaService when installing JOLIE with ant install. Just type metaservice -h and a help screen will welcome you.

The script supports passing MetaService's input port location and protocol as parameters. This means that for KDE launching MetaService just became a matter of executing something like:

metaservice localsocket:/tmp/ksocket-$USER/metaservice

Hooray for simplicity!
January 20, 2009
The Jolie web site now sponsors a "Jobs List", which can be found from the Contribute page, also known as "What we lack for world domination". The list features some Junior Jobs, so it's a good starting point for people that are looking for ways to start contributing to Jolie. We are continuously filling in new jobs (there's *always* something to do, as in every project ;), so be sure to refresh it if you're interested. If you don't find anything that interests you, remember that you can always post to the mailing list. =)

Now that JOLIE supports message types, it is feasible (and not too hard) to make automatic generators for Plasma::Service description files. These jobs are listed, so if someone wants some cool tool for automatic type-safe code generation between Plasma and JOLIE sooner rather than later and to learn something more about how Plasma and JOLIE communicate, this is a good occasion to jump on the JOLIE<->Plasma revolution train. ;)
January 19, 2009
For everyone using Windows out there: compiling JOLIE from sources in Windows is not a pain anymore!

I've patched the build system so that it recognizes the running operating system and installs the appropriate launcher script (a shell script in *nix and a .bat script in Windows), so now having a working environment is far less "manual" than before. You can find the updated guide here (Windows -> From sources).
January 12, 2009
JOLIE has a new and shiny website: www.jolie-lang.org

The new site has a wiki which we are filling in with a lot of information, minute by minute. Some tutorials are already final and we are preparing demos and packages of source code examples.

This step required some effort and time, but we now have a serious and stable basis to build our knowledge base upon (the backend of the website is quite powerful and allows for easy administration and data versioning).

The mailing list and IRC channel are already active since some time, respectively jolie-devel@lists.sourceforge.net and #jolie on freenode, providing the means for asking about anything related to JOLIE and getting the attention of the JOLIE team.
October 10, 2008
Today, MetaService-java hit Jolie's SVN. You can find it in /trunk/support/metaservice-java. Basically, MetaService-java is a Java API abstraction layer for interacting with MetaService, so that you can use it as if it were a Java object.

This project was born for two reasons:
- it will be part of a solution for the integration of Java enterprise web applications with Service-Oriented Computing;
- it is a good example of how to implement an API abstraction layer to MetaService from an Object-oriented language, so its source code could be a useful reading for the Plasma::Service developers.

I'll let the code (and its comments) and a simple example speak for themselves. They're in Java, but they should be pretty easy to understand even for people non-proficient with it. Example follows.


MetaService metaService = new EmbeddedMetaService(); // Create a MetaService instance.

// Set up access to a SOAP Web Service.
MetaServiceChannel myWebService = metaService.addRedirection(
"MyWebService", // Resource name to assign
"socket://www.mywebservice.com:80/", // Service location
Value.create( "soap" ), // Protocol to use
Value.create( "My metadata" ) // Descriptive metadata
);

// Done! Let's communicate with it.
myWebService.send( "getNameById", Value.create( 4 ) );
Value response = myWebService.recv();
System.out.println( response.strValue() ); // Will print the name.


The API is still to be refined, but pretty much usable already. =)
October 02, 2008
This fall looks good so far! A lot of JOLIE sub-projects are taking shape, and are going better than anticipated. But I'm blogging about something which has just landed on JOLIE's svn.

JOLIE now supports local sockets (a.k.a. unix sockets). How to use them? Let's see it by comparison; a communication interface over tcp/ip sockets is exposed like the following:

inputPort MyInputPort {
Location: "socket://localhost:9000"
Protocol: soap
Interfaces: MyInterface
}


Let's switch that to a local socket:

inputPort MyInputPort {
Location: "localsocket:/tmp/mylocalsocket"
Protocol: soap
Interfaces: MyInterface
}


That's it! Use the localsocket scheme in the Location URI, and supply the right path to it.

August 12, 2008
Frank's talk was great: bringing the community into the KDE desktop is actually a great idea.
Even better, he's building up REST services for accessing the data! Of course, as we're speaking of services, my mind went directly to "Hey, these things are Joliable".

I've already started collaborating with Frank in order to discuss the API and a set of specifications for publishing a machine-readable format for reaching said services.

While I'm at it, someone could wonder "Just how difficult is to use these things with JOLIE?". The answer, of course, is "damn easy" (screencast). ;)

So, once we're finished with the JOLIE<->Plasma integration, expect nice graphical frontends to pop up. =)
August 09, 2008
The more I speak with people at Akademy, the more I'm convinced that contacting KDE to start the JOLIE<->Plasma integration thing has been the right decision.

It looks like the whole KDE community (users and developers) is a perfect match to start "porting service-oriented computing to the masses". There is an incredible need for integration, communication, sharing and service accessibility. And when I see talks like the one given by Frank Karlitschek (KDE Community websites: The past, the present and a vision for the future) I get the feeling that we are all trying to converge to a service-oriented experience. We have the right tools, let's make that happen!

I do really hope to clarify what this means and how we can do it with my presentation this afternoon, so... here I am checking that the demos are still working. Murphy's law, begone! ;)
August 06, 2008
Phew, the temperature is so high here in Bologna that it's difficult even to think.
Nevertheless, this month is gonna be even hotter! So, here are some quickies of the current things that are happening.

Akademy, Akademy, Akademy...
I'm finishing my preparations for Akademy, and the presentation is coming out nicely. I'm trying to make it understandable even by people who does not know much about Service-oriented Computing. I will also demo a couple of service-oriented applications for KDE (of which you could have seen screencast/shots in the kde blogs), and explain how they work and how easy is to make them with JOLIE.
By the way, if you need/want to talk with me, I'll be at Akademy the 9th, the 10th and the 11th (the 11th only in the morning).

Rising interest
We, as italianaSoftware, are receiving more and more interest in JOLIE by the business world. It looks like this could produce some open-source products. We shall see what comes up. The industry involvement has already produced some good things, like the sponsoring of the Google Web Toolkit integration in JOLIE (we are already using this in the open-source world, too: Echoes is based upon the JOLIE-GWT compatibility).

A lot of features to come in the next months
Some projects from the academic side are taking shape. A student from the University of Bologna, Davide Malagoli, is putting the basis for a complete bi-directional JOLIE<->XML translator. While JOLIE already supports the SOAP protocol, this is a great thing for the integration with tools designed for the Web Services technology. For instance, we will be able to generate automatically a WSDL document of a JOLIE program. As JOLIE supports more than just SOAP and TCP/IP sockets, we are already designing standard compliant WSDL elements to describe that.
We have got other two projects in early development, I will blog about them in the near future.

Looks like this fall and winter I will have a lot of features to put in the source tree, and a lot of screencast/shots to make. =)
As for this coming week-end, see you all at Akademy!


P.S.: welcome back to the blog world, Aaron!

UPDATE! Got the permission from the student to blog his name, so here it is. ;)
July 24, 2008
Hello, readers.
So here I am, another piece in the blogosphere. This blog will be about the development of JOLIE, a language for service-oriented programming, and all of its related works.

As of this first post, this blog is on Planet KDE; the following says why.

If you read the planet, you should have seen JOLIE cited here and there already. Basically I'm on Planet KDE because we, the JOLIE and KDE (Plasma especially) projects, are cooperating in order to bring the power of service-oriented computing (see also: service-oriented architecture, service-oriented programming, etc.) to the desktop.
This blog of mine is here also to explain what this means and will mean for the KDE desktop, and to show the practical results of this effort. Stay tuned with my blog if you want to know more!
July 24, 2008
Vision is a proof of concept application (thanks aseigo for the initial suggestion), showing what can be easily obtained by using the service-oriented paradigm together with rich client applications.

What is Vision?
But, more importantly: what's a distributed presentation?

Imagine that you are showing a presentation, and that someone wants to see what you're presenting... in his computer! To do so, he should connect in some way to your presentation. Whenever you change page (or make another relevant action, like changing document), the client should be notified and synchronized automagically.
That is exactly what Vision does. It forms a network between the presenter and the clients, keeping the clients synchronized with the presentation.
A screencast is better than a thousand words: an example usage of vision with the previewer plasmoid and kpdf.

As you can see in the screencast, Vision already supports more than one viewer (there are shown kpdf and the previewer plasmoid, but we also already support okular).

Another nice feature of Vision is that it forms a P2P network. A client acts also as a presentation server. Say that A is watching B's presentation. Now another client (C) can connect to A. Whenever A receives an event from B, it will now also send it to C.
This can be exploited, for example, in order to perform bandwidth load balancing.

Note also that Vision makes use of SODEP, a binary protocol especially designed for service communications (it can, nevertheless, use all of the other JOLIE supported protocols, such as SOAP, HTTP, GWT-RPC...). Thus, the needed bandwidth is very small and the application can handle a lot of clients at the same time.


Exposing Vision to other platforms
A great feature of JOLIE is that it separates the logical interfaces of a service from their deployment. Suppose that we want to extend Vision to support Bluetooth clients. The ideal solution would be to expose the same interface that we expose on the network via sockets to bluetooth. Luckily, doing that in JOLIE is very easy:


service PresenterBluetoothService
{
Location: "btl2cap://localhost:3B9FA89520078C303355AAA694238F07;name=Vision;encrypt=false;authenticate=false"
Protocol: sodep { .charset = "ISO8859-1"; .keepAlive = 1 }
Ports: PresenterInputPort
}


What do these lines mean?
Well, they're telling JOLIE to expose the PresenterInputPort interface (which is the Vision presenter interface) on the pointed bluetooth Location using the SODEP protocol to encode and decode data. You can see the whole presenter service code in the JOLIE SVN repository (/trunk/playground/vision).

Guess what? It just works! (video) And what's better than a mobile phone to test it?

By the way:


just in case you want to see more service-oriented goodness in KDE. =)