This is a simple example of code to demonstrate how to use the library fakesmtp-junit-runner to write integration tests.
The library has been released on GitHub and Maven Central. Continue Reading
Writing an integration test with a mail server
integration
TOP open-source dashboard solutions 2017
Dashboards can be a very efficient communication tool for a team, between managers and business units. It enables an organization around a vision to share common goals. It can also be useful to identify weaknesses in processes and adapt your strategy according to them.
SonarQube and ReactJS
This article is showing you how to use SonarQube with ReactJS and its JSX files. I will use both SonarQube JavaScript plugin and the additional plugin Sonar EsLint plugin.
<%- toc(str, [options]) %>
For the people who has missed my previous article, I have created a new SonarQube plugin to extends the Javascript analysis.
Installation and Configuration
The first step is to download the plugin directly from Github here.
Download the plugin
Find the latest release.
Find the latest release
Copy it in your Sonar extension folder.
Copy the plugin
Restart the server
Restart the server by calling the commands (here on linux)
➜ sonarqube-6.0 ./bin/linux-x86-64/sonar.sh stop
Stopping SonarQube...
Waiting for SonarQube to exit...
Stopped SonarQube.
➜ sonarqube-6.0 ./bin/linux-x86-64/sonar.sh startEnabling custom rules in SonarQube
Don’t forget to modify your SonarQube profile to enable the new ESLint rules :
Add the ESLint rules to your SonarQube profile
Enable the ESLint rules to your SonarQube profile
Preparing your project
Handling SonarQube Scanner
Most projects requires the SonarQube scanner (Wiki Link to analysis Javascript. Download it somewhere on your disk and unzip it.
Creates a file sonar-project.properties̀ into your project.
Copy-paste this content and modify it :
sonar.projectKey=sleroy:reactjs-demo
sonar.projectName=ReactJS demo
sonar.projectVersion=1.0
sonar.sources=src
sonar.sourceEncoding=UTF-8
sonar.javascript.file.suffixes=.js,.jsxDon’t forget the line sonar.javascript.file.suffixes=.js,.jsx, it’s the hack to make SonarQube working on JSX files!
OK! SonarQube Scanner is configured!
Preparing ESLint
We want to perform the SonarQube analysis with the additional results of ESLint. Eslint is a popular linter that provides recent rules for many javascript frameworks – ReactJS included.
ESLint is thereby often upgraded and contains through its extension system, rules and frameworks that you won’t find in the regular SonarQube installation.
If you haven’t created yet an ESLint configuration file, here is the commands :
ESLint Configuration
You can try the configuration by launching ESLint ony your project. It may warn you that some extensions are missing. Install them with NPM or Yarn.
Missing NPM Module
Usually, the ReactJS extension is missing of your project. You can add them like developer extensions (--save-dev) or globally (-g).
Install missing ESLINT ReactJS extension
With the right configuration and ESLint installation, the scan of a JSX File should work :
Scanning JSX File
OK! ESLint is configured!
Launching SonarQube Scanner
Launchs the SonarQube scanner with the command :
~/tools/sscanner/bin/sonar-scannerAnd the analysis is running …
➜ react-jsx git:(master) ✗ ~/tools/sscanner/bin/sonar-scanner
INFO: Scanner configuration file: /home/sleroy/tools/sscanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /home/sleroy/git/react-jsx/sonar-project.properties
INFO: SonarQube Scanner 3.0.3.778
INFO: Java 1.8.0_121 Oracle Corporation (64-bit)
INFO: Linux 4.10.0-21-generic amd64
INFO: User cache: /home/sleroy/.sonar/cache
INFO: Load global repositories
INFO: Load global repositories (done) | time=211ms
INFO: User cache: /home/sleroy/.sonar/cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=14ms
INFO: SonarQube server 6.0
INFO: Default locale: "fr_FR", source code encoding: "UTF-8" (analysis is platform dependent)
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=214ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=94ms
INFO: Load active rules
INFO: Load active rules (done) | time=897ms
INFO: Publish mode
INFO: ------------- Scan ReactJS demo
INFO: Load server rules
INFO: Load server rules (done) | time=482ms
INFO: Base dir: /home/sleroy/git/react-jsx
INFO: Working dir: /home/sleroy/git/react-jsx/.scannerwork
INFO: Source paths: src
INFO: Source encoding: UTF-8, default locale: fr_FR
INFO: Index files
INFO: 9 files indexed
INFO: Quality profile for js: Sonar way
INFO: Sensor Lines Sensor
INFO: Sensor Lines Sensor (done) | time=41ms
INFO: Sensor SCM Sensor
INFO: SCM provider for this project is: git
INFO: 9 files to be analyzed
INFO: 0/9 files analyzed
WARN: Missing blame information for the following files:
WARN: * /home/sleroy/git/react-jsx/src/example/hello.jsx
WARN: * /home/sleroy/git/react-jsx/src/example/index.jsx
WARN: * /home/sleroy/git/react-jsx/src/fixtures/this.jsx
WARN: * /home/sleroy/git/react-jsx/src/example/index.js
WARN: * /home/sleroy/git/react-jsx/src/example/imager.jsx
WARN: * /home/sleroy/git/react-jsx/src/fixtures/component.jsx
WARN: * /home/sleroy/git/react-jsx/src/fixtures/advanced.jsx
WARN: * /home/sleroy/git/react-jsx/src/fixtures/react.jsx
WARN: * /home/sleroy/git/react-jsx/src/fixtures/hello.jsx
WARN: This may lead to missing/broken features in SonarQube
INFO: Sensor SCM Sensor (done) | time=504ms
INFO: Sensor XmlFileSensor
INFO: Sensor XmlFileSensor (done) | time=1ms
INFO: Sensor JavaScript Squid Sensor
INFO: 9 source files to be analyzed
INFO: 9/9 source files have been analyzed
INFO: Unit Test Coverage Sensor is started
INFO: Integration Test Coverage Sensor is started
INFO: Overall Coverage Sensor is started
INFO: Sensor JavaScript Squid Sensor (done) | time=893ms
INFO: Sensor Linting sensor for Javascript files
INFO: Sensor Linting sensor for Javascript files (done) | time=1438ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=38ms
INFO: Sensor Code Colorizer Sensor
INFO: Sensor Code Colorizer Sensor (done) | time=2ms
INFO: Sensor CPD Block Indexer
INFO: DefaultCpdBlockIndexer is used for js
INFO: Sensor CPD Block Indexer (done) | time=1ms
INFO: Calculating CPD for 2 files
INFO: CPD calculation finished
INFO: Analysis report generated in 170ms, dir size=24 KB
INFO: Analysis reports compressed in 254ms, zip size=18 KB
INFO: Analysis report uploaded in 39ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard/index/sleroy:reactjs-demo
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://localhost:9000/api/ce/task?id=AVwHr7JyDHBkCqlFC7Sx
INFO: Task total time: 8.046 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 10.141s
INFO: Final Memory: 48M/301M
INFO: ------------------------------------------------------------------------Controlling the results
Go to your Sonar interface, and jump directly to the dashboard.
Our project has been analyzed.
SonarQube analysis
We observe that the violation’s number is increasing with the new rules.
SonarQube analysis details
Hourra, our JSX files are analyzed !
JSX Analysis
In this article, we have installed, configured a new plugin to perform better Javascript analysis into SonarQube, working with ReactJS and JSX files.
Release of FakeSmtp-junit-runner
Today, I released a new library to help developers to write integration tests with mail servers.
The library has been released on GitHub and Maven Central.
fakesmtp-junit-runner
Links : github.
Important : Part of the source code of this library has been modified and adapted from the project of FakeSmtp. I want to thank him since his project inspired me the creation of that library.
This library is an extension to JUnit to allow developers to write integration tests where a SMTP server is required.
The how-to is quite simple :
- Inserts the @Rule in your integration tests
- a Fake SMTP Server will start
- You can send mails on it
- You can control the mailbox
- Write your own assertions to check mails.
Installation
The project requires JUnit 4.11 or higher. It also requires SLF4J API presents in the classpath. I did not bundle them in the library to avoid conflicts.
To use it, adds the library to your maven or gradle config script :
For maven :
<dependency>
<groupId>com.github.sleroy</groupId>
<artifactId>fakesmtp-junit-runner</artifactId>
<version>0.1.1</version>
<scope>test</scope>
</dependency>For gradle :
testCompile "com.github.sleroy:fakesmtp-junit-runner:0.1.1"Usage
Step 1 :
Creates a JUnit test :
public class SmtpSendingClassTest {
@Test
public void testCase1() {
}
}Step 2 :
Adds the new Junit rule with its configuration :
public class SmtpSendingClassTest {
@Rule
public FakeSmtpRule smtpServer = new FakeSmtpRule(ServerConfiguration.create().port(2525).charset("UTF-8"));
@Test
public void testCase1() {
}
}Step 3 :
You are ready to use it, controls the mailbox or the server state :
Assert.assertTrue(smtpServer.isRunning());public class SmtpSendingClassTest {
@Rule
public FakeSmtpRule smtpServer = new FakeSmtpRule(ServerConfiguration.create().port(2525).charset("UTF-8"));
@Test
public void testCase1() {
Assert.assertTrue(smtpServer.isRunning());
Assert.assertTrue(smtpServer.mailbox().isEmpty());
}
}How migrate from JBoss 5 to 7
This article is part of my working notes on the subject of “How to migrate Web applications running on JBoss AS 5 to the version 7”.
JBoss Application Server – Wildfly
I will go straight to the details though here is some lines about JBoss Server.
From Wikipedia : JBoss Application Server(Now called Wildfly) is an application server authored by JBoss, now developed by Red Hat. WildFly is written in Java, and implements the Java Platform, Enterprise Edition (Java EE) specification. It runs on multiple platforms.
On 20 November 2014, JBoss Application Server was renamed Wildfly.
The product history according Wikipedia is :
- 5.1 Release 23 May 2009
- 7.0[10] Release 12 July 2011
- 7.1 Release February 2012
- 10.1.0 Release August 2016[20]
The JBoss AS community project has been renamed to the WildFly community project wildfly.org
According this JBoss 5 to 7 in 11 steps, the benefits are :
Processing time decreased by 25% without any code change. Development speed increased in my opinion (it is really hard to measure it) by 50% and we are much more productive (faster server restarts). Memory footprint lowered from 1GB to 512MB. Finally automatic application redeployment finally works! However there is always a price to pay – the migration took us 4 weeks (2 sprints).
Thanks to the presentation from Roberto Cortez, we have a clear picture of the migration.
[slideshare id=54488564&doc=migrationtalesfromjavaee5to7-151028171122-lva1-app6892]
JBOSS 5 Architecture
JBOSS 7 Architecture
The checklist
Prepare the checklist
When the PAAS or the Web application server have to be upgraded, several regressions may happen. The team has to pay attention to :
- Server functionalities and integration : Performance, Security, Logging, Monitoring
- Server configuration
- Server deployment configuration
- Application deployment configuration
- Server API regressions
- Application regressions
- Training and risk management
Server functionalities and integration : between the versions, some functionalities and integrations provided by the server may have evolved, be fixed or simply disappeared.
Server configuration : The way the server has been configured, using scripts, GUI, may have changed, forcing the team to change their configuration files and finding the corresponding new way of doing it.
Server deployment configuration : Your deployment model configuration may have to be upgraded : single node, clustered mode, disaster recovery, high availability, reverse-proxying may behave differently in the new versions.
Application deployment configuration : the way to deploy your web applications may have changed in the new versions (GUI mode to script mode…)
Server API Regressions : usually Web Application servers are implementing a specific JEE API version, Servlet API and so on. These API may have changed causing regressions in your applications.
Application regressions : JBoss is including many components extending the JEE with BPM, Persistence, implementation. It is really important to track your dependencies (using Tattletale or mvn dependencies:tree) and interview your team about possible hacks and fixes to overcome the limits of JEE 5. This kind of workaround is difficult to migrate.
Training and risk management : This kind of migration contains its part of risks and changes. Both can create frictions inside your team of between the IT Team and your Dev teams. To ease the migration, don’t forget to dedicate some time to your teams into training to learn the new features of JAS 7. You will also have to adapt your project management to freeze for a while the features until the migration has been done.
Global checklist
This section is providing a checklist to help developers and managers to evaluate the migration risk of their applications.
Common issues
Here is a list of common issues during the migration of applications with JBOSS AS.
4.2. Debug Migration Issues 4.2.1. Debug and Resolve Migration Issues 4.2.2. Debug and Resolve ClassNotFoundExceptions and NoClassDefFoundErrors 4.2.3. Find the JBoss Module Dependency 4.2.4. Find the JAR in the Previous Install 4.2.5. Debug and Resolve ClassCastExceptions 4.2.6. Debug and Resolve DuplicateServiceExceptions 4.2.7. Debug and Resolve JBoss Seam Debug Page Errors
- There is a deadlock when using EJB remoting over SSL. This deadlock is present even in EAP 6.2. We’re now at the point when we have quite a patch set of features backported from WildFly to AS 7.
- JMS : JBoss Messaging server has been deprecated and the compatibility with a JBOSS AS 5 server is really tough to maintain. Some solutions exists as explained below in the article.
What is changing ?
Here is a summary of the evolutions between the JBoss AS version 5 and the version 6.
JBOSS AS 6 changes
Here is the compiled list of modifications including the minor version fixes.
- [Server functionality, Server API, Application regressions] Module based class loading
In JBoss Enterprise Application Platform 5, the class loading architecture was hierarchical. In JBoss Enterprise Application Platform 6, class loading is based on JBoss Modules. This offers true application isolation, hides server implementation classes, and only loads the classes your application needs. Class loading is concurrent for better performance. Applications written for JBoss Enterprise Application Platform 5 must be modified to specify module dependencies and in some cases, repackage archives.
- [Server functionality, Server deployment configuration] Domain Management : In JBoss Enterprise Application Platform 6, the server can be run as a standalone server or in a managed domain.
- [Application Deployment] Deployment Configuration : Standalone Servers and Managed Domains : Boss Enterprise Application Platform 5 used profile based deployment configuration. These profiles were located in the EAP_HOME/server/ directory. Applications often contained multiple configuration files for security, database, resource adapter, and other configurations. In JBoss Enterprise Application Platform 6, deployment configuration is done using one file. JBoss Enterprise Application Platform 5 configuration files must be migrated to the new single configuration file.
- [Server functionality, Application deployment configuration] Ordering of deployments : Application Platform 5 applications that consist of multiple modules deployed as EARs and use legacy JNDI lookups instead of CDI injection or resource-ref entries may require configuration changes.
- [Server functionality, Application deployment configuration] Directory Structure and Scripts : As previously mentioned, JBoss Enterprise Application Platform 6 no longer uses profile based deployment configuration, so there is no EAP_HOME/server/ directory.
- [Server application, Application deployment configuration, Application regression JNDI Lookups JBoss Enterprise Application Platform 6 now uses standardized portable JNDI namespaces.
- [Server functionality, Server configuration, Application configuration, Application code] : Changing logging dependencies
- [Server API, Application configuration regressions] Resource adapter configuration : In previous versions of the application server, the resource adapter configuration was defined in a file with a suffix of *-ds.xml. In JBoss Enterprise Application Platform 6, a resource adapter is configured in the server configuration file.
- [Server libraries] Technologies upgrade : JDK 6, JSF 2, Bean Validation (JSR-303), CDI, EJB 3 (1.1.13)
- Include mod_cluster
- Servlet API 3.0
- Update CL to 2.0.8.GA
- Update Deployers to 2.0.9.GA
- Update Javassist to 3.11.0.GA
- Update JBossWS to 3.2.1.GA
- Update JBossXB to 2.0.2.Beta3
- Update JGroups to 2.6.13
- Update Kernel to 2.0.9.GA
- Update MC-INT to 2.2.0.Alpha2
- Update MDR to 2.0.2.GA
- Update to Entity Manager 3.5 and JPA 2
- Update to JBoss AOP 2.1.6.GA
- Update VFS to 2.2.0.Alpha1
- Upgrade apache-beanutils to 1.8.0
- Upgrade ha-server-cache-jbc to 2.1.0.GA
- Upgrade JBoss Cache to 3.2.1.GA
- Upgrade jboss-common-core to 2.2.16.GA
- Upgrade jboss-ha-server-cache-jbc to 2.0.1.GA
- Upgrade JBoss JAXR to 2.0.1
- Upgrade JBoss LogManager to 1.1.0.GA
- Upgrade JBoss Security 2.0.4.SP2
- Upgrade JBossXACML to 2.0.4
- Upgrade JSF to 2.0.0-RC
- Upgrade to Java Mail 1.4.2
- Upgrade to JBossXACML 2.0.3.SP2
- Upgrade XNIO Metadata to 1.0.1.GA
- New library JBossWS-CXF
- library update RestEasy
- JBoss Messaging JMS & MDB replaced by Hornet MQ
- New RMI Framework : Remote 3
- VFS Library update
- [Server functionalities] new server functionalities : Mod_cluster, JBoss Embedded AS
- [Application deployment] The legacy pooled invoker has been removed. Applications using the pooled invoker should switch to the JBoss Remoting-based unified invoker, which has been the default detached invoker since 4.2.
JBOSS AS Release 7
Here is the compiled list of modifications including the minor version fixes.
- [Server functionality, Application regressions] Security improvement
Unlike previous releases, with AS 7.1, remote access requires secure authentication by default. This includes both managment (native, jmx, etc) and various remote application protocols (ejb, jndi, jms, etc); Added SSL support for the Remoting interfaces.
- [Application configuration deployment] Management API improvements : All configuration attributes are updatable via the CLI. Direct edits to the XML are not necessary.
- [Server functionality] Various Administration Console Improvements and Management changes
- [Server API] Remote Connectivity Added support for remote EJB, JNDI and JMX invocation over JBoss Remoting 3, IIOP, Remote JMS. Three modes for accessing remote EE components using JNDI (Client, Traditional Remote, and Delegated).
- [Server deployment model] Clustering Enhancements : Standalone Servers and Managed Domains : Numerours fixes in HTTP Session Replication, Clustered Web SSO, EJB Stateful Session Bean Replication, EJB load-balancing and failove, JPA XPC propagation
- [Server functionality] CLI Regressions jboss-admin.sh renamed to jboss-cli.sh,
data-source add" "--pool-name"argument seems to have changed to “–name”. - [Server libraries] Technologies upgrade :
- EJB 3.1 Full – Adds a number of key features, including remote communication, asynchronous method invocation, timers, message-driven beans, and legacy compatibility with EJB 2.
- CMP 2 – Provides a legacy persistence manager which predates JPA. This is benefical to legacy applications which make use of EJB 2.x Entity Beans.
- JAX-WS 2.2 – Allows simplified usage of Web Services in the EE platform.
- JAX-RPC 1.1 – Offers legacy support for older Java EE Web Services applications.
- JAX-RS 1.1 – Supports the construction of RESTful Web Services using the Java EE platform.
- JavaMail 1.4 – Allows Java EE applications to send and receive e-mail
- JCA 1.6 – Provides a mechanism for third parties to provide support for custom data sources, as well as connection pooling and transaction management for database access.
- JMS 1.1 – Adds advanced messaging support to EE applications.
- IIOP – Supports interoperablility with other application servers and non-Java CORBA clients.
- JSR-88 – Allows for managing deployments to a Java EE server in a portable fashion.
- Update mod_cluster to 1.2.0.Final
- IronJacamar 1.0.7.Final
- Upgrade Infinispan to 5.1.0.CR3
- Upgrade to JBossTS 4.16.1
- Upgrade jboss-metadata to 7.0.0.Beta33
- Upgrade JGroups to 3.0.3.Final
- Upgrade JBoss Marshalling to 1.3.6.GA
- Upgrade httpcore to 4.1.4
- Upgrade to JBossWS 4.0.1.GA and Apache CXF 2.4.6
- Update to classfilewriter 1.0.1
- Upgrade to JSF 2.1.7
- Upgrade PicketLink to 2.0.2.Final
- Upgrade PicketBox to 4.0.7.Final
- Upgrade commons-beanutils to 1.8.3
- Upgrade Google Guava to 11.0.2
- [Server functionalities] new server functionalities : Mod_cluster, JBoss Embedded AS
- [Application deployment] The legacy pooled invoker has been removed. Applications using the pooled invoker should switch to the JBoss Remoting-based unified invoker, which has been the default detached invoker since 4.2.
- [Server functionalities] new server functionalities : Mod_cluster, JBoss Embedded AS
- [Application deployment] The legacy pooled invoker has been removed. Applications using the pooled invoker should switch to the JBoss Remoting-based unified invoker, which has been the default detached invoker since 4.2.
How to migrate : plan and tasks
- JDBC configuration
- Classpath references
- Global Modules Reference
- JMS migration :
According link, we thought it would be really hard to connect with JMS server based on JBoss 5. It turned out that you have 2 options and both work fine:
- Start HornetQ server on your own instance and create a bridge to JBoss 5 instance
- Uses a JMS Bridge to move the existing messages
- Use Generic JMS adapter: https://github.com/jms-ra/generic-jms-ra
Application packaging and configuration
- Repackaging Dependencies and fix the EAR Layout: link
- Install and configure the JDBC Driver link
- Update the Resource Adapter Configuration
- Configure the datasource for Hibernate and JPA : If your application uses JPA and currently bundles the Hibernate JARs, you may want to use the Hibernate that is included with JBoss Enterprise Application Platform 6.
In WildFly 8, a resource adapter is configured in the server configuration file. If you are running in domain mode, the configuration file is the domain/configuration/domain.xml file. If you are running in standalone mode, you will configure the resource adapter in the standalone/configuration/standalone.xml file.
More details there How to migrate from AS5 or AS6 to Wildfly
- Migration of the shell scripts, integration test scripts, deployment scripts.
Application code and configuration migration
Here is the list of tasks implying some rewriting inside your application code.
- Migration JEE 5 to JEE 6
- Upgrade to JPA 2.0
- Update your SOAP Implementations using JBossWS-CXF
- Upgrade Hibernate from 3 to 4
- Replacing JBOSS Cache by Infinispan Cache
- Configure JAX-RS / Resteasy changes
- Fix Hibernate’s sequencer
- Replace JBOSS AOP Interceptors : JBoss AOP was used by the EJB container. However, in AS 7, the EJB container uses a new mechanism. If your application uses JBoss AOP, you need modify your application code as follows.
- Migrating JNDI : Migrating JNDI namespaces
- Update the datasources : link
- Rewriting of your RMI Code : oss 5 and 7 are totally different and this kind of communication will not work.
- Using CDI instead of plain old Singletons
Tooling
Here is a list of useful tools to assist you in your migration.
- IronJacamar : to update your datasource configuration
- Tattletale : to find the application dependencies
- Gilder : An application for migrating the configuration of JBoss AS 5-based servers to JBoss AS 7-based servers.
- Tools and tooling to migrate to JBOSSS 6 : Link
- Upgrades to newer versions of WildFly or JBoss EAP may be handled using the JBoss Windup migration tool. JBoss Windup migration tool
References:
- https://www.youtube.com/watch?v=eelLC2kF9bU
- JBoss 6 release
- JBoss 5 to 7 in 11 steps
- JBoss Release note 6
- JBoss Enterprise Application Details
- PDF Migration JBOSS
- JBOSS 5-7
- JBoss Server migrqtion tool
- How do I migrate my application from AS5 or AS6 to AS7
- JBOSS 5 to 7
- Migration tales JEE 5 to 7
How Docker is disrupting Legacy IT Companies
Thanks to its popularity, Docker has disrupted many companies and blurred the silos between Developers and Operations. In this article, based partially on my own experiences, I will depict some of the disruptions that containers have provoked into the IT companies. I wish that this article will depict familiar situations and brings you argument to win the obstacles to the container technology propagation 🙂
Disclaimer : Despite I am quoting Docker quite a lot in this article, it is not an endorsed article. If you have a better alternative, simply replace Docker by another Container Technology, the arguments should still be valid.
If you appreciate this article, please relay or like it.
Day one : I don’t need to spend four hours to setup my development environment
That is my first day on the job. A brand-new laptop, decent performance and feature. Default Operating system : Windows.
Well, I am a Linux hardcore developer and I spent many efforts to live without Excel, PowerPoint and Outlook. And who knows ? Maybe my customers won’t be on Windows. Therefore, I am wondering how to create my new software development environments ? Node.JS ? Java ? Mobile, each technology is coming with their own tools, servers and configuration.
What are the choices ? The company is fair and provides a decent laptop with sufficient power. However the software installation on the native OS has been blocked. I need to proceed by virtual machines. Virtual machines ? What a cumbersome solution. I need to download ISO or OVA’s and proceed to the installation of my software. What is your virtual machine creation’s strategy ? I made a quick survey to my colleagues, who confirm that they create a virtual machine by customer’s project. And they share their virtual machine like Pokemon. WTF ? Many dozen of gigabytes are transferred through USB3, a tiny hard drive and the team is ready. Well after several hours.
Docker or any similar container technology is offering me a better solution.
These are the following arguments :
-
Reduce your startup time and be more efficient. You can find many docker images to set up a development environment ready to use :
- Docker image node.js dev : A Dev environment for JS
- Docker image Ruby Dev, Docker image Ruby Dev2
- Docker image C/C++ on Linux
- Docker image Java Dev
-
Broadcast your software programming best practices by using the same env in your team. As a tech lead, my mission is to make my colleagues better than me. And to reach that goal, I am trying to provide them the best tools, configuration, IDE, and automation to help them in their work. How many times, I had to provide a format style guideline to indent their code ? A syntax checker configuration ? An IDE with the right plugins ? All these issues can be solved by providing my docker image and updating it regularly.
-
The time for for Web IDE software is probably come : Eclipse and Visual Studio, Borland Delphi, such IDE have been used by generations of developers. They all come with the same advantages and drawbacks. Powerful, clever code completion, nice OS integration and notifications, a whole bad of features. Clearly the develop has a great environment to write its software but these solutions do not scale well inside a team. How to share my configuration ? My preferences ? How to share code ? How to communicate ? To create coziness in your team, you will have to rely on a great IT administrator. A magician of the command line, Powershell to setup your OS with the same configuration everywhere, yet able to update it regularly.
My recommendation is to rely on two kind of tools to produce your software : – lightweight code editors such like Atom, Visual Studio Code from Microsoft – Web IDE such like Cloud 9 or CodeEnvy is a great example. Using Eclipse Che, the web rewrite of Eclipse, a Saas IDE with the same lot of features and configuration. The most amazing thing is that this great and complex system can be installed with a Docker one liner.
Day two : Security everywhere, freedom and performance nowhere
As software specialists, we are well aware of the threats coming from the web applications, unobserved operating systems, data breaches. Our daily duty is to protect our customer data.
The consequence for developers is a matter of rule, our computers are locked. Software installation is double checked by IT, Internet is accessed through a proxy, antivirus, whitelist and so on. Hard disk ciphered and so on. in ch All developments have to be done on Virtual machines. BUT Virtual machines are such a pain to manage. A huge disk space, hard to customize, fairly expensive solutions (licence cost of VMWare to be able to perform VM snapshot on every developer laptop…)
Docker Datacenter
Building Virtual Disk images is a tremendous task for the system administrators : slow to copy, hard to customize, mostly manual installation and snapshots to produce them. Developers does not find the necessary flexibility to adapt to their customer projects.
Docker is offering a neat and efficient way to produce images, thanks to the docker script language. A good mix between automation and the traditional system administrator work of producing shell scripts.
Using Docker images is offering enough security control to system administrator, less efforts to maintain and developers can also provide easily their own images to the Security/Administrator Team for review. But how to store your Docker images. At the present time, I would recommend something like Docker Datacenter to host your company containers on premise.
Day twenty : The typical legacy IT Project
Traditional "legacy" IT Projects features : * a code base * scripts to build the software * some manual test cases * a huge and extensive installation documentation to setup : * the test environment * the production environment * perform the maintenance, the upgrade, the backup of the system * scripts to install the database schema
In practice, most IT projects enforces developers to manually install their development, test, production environments using an out of date documentation, incomplete. How the software team is performing a QA session ? Will they create a brand new test environment with fresh data in a given state and the latest produced software version ? The answer is probably no, definitively no.
Usually, IT teams are relying on a single test server, painstakingly built through the scrum sprints, on a virtual machine. Do you think that I am exaggerating ? Ask to your team, how much time do they need to recreate this environment ? And what if their snapshot is lost or damaged ?
Continuous Dockery, ElectricCloud image property
Docker is providing several solutions to common IT project problems :
-
Deploying the customer application on the developer laptop : docker images are shared between developers to have access to a debug environment. Docker composer can help the software development team to build ready-to-use.
-
Initializing and populating a database for tests : another solution to execute integration tests is to rely on Docker to build an image, ready-to-use of your data. Start the container, wait the readiness , execute your integration tests and kill the container once used. Such scenario is easy to create with Docker, even with proprietary databases such like Oracle of MSSQL. This article is a good instruction to Docker and test automation.
Deterministic Test Automation
- Multiple target and environment compilation : Developers often need to test their software in different environments, browsers. Docker images also provide a solution to the complexity of a software environment.
Day forty : The void of the production environment
Recently, I have encountered a brilliant developer – although alone – maintaining a messy piece of PHP code. He was not the originator of the project, though, in charge of the project since two years. He told me that the manager offered him a virtual machine with everything on it at the begin of the project, to help him. The same one he is using on it.
Currently, he is struggling with the customer and that software. Both the customer and him have different deployment environments. And the difference of server, languages and frameworks versions is creating a huge mess.
Another project and situation. This IT team has been relying on Ansible (with Puppet it would have been the same situation), to deploy their software in the different environments. Despite the improvements thanks to the automation provided by Ansible, there is always a slight tension when launching the Ansible scripts. Maybe it’s the system entropy, the virtual machine erosion, most likely the reason is that the virtual machine has never been deleted and recreated. Anyway there are some subtile differences between the environments and probably and the Ansible deployments are sometimes failing when new features are shipped.
System erosion
With that team, we have reached a common point of view. When should we use Ansible to deploy the software ? We should rather use Ansible to prepare the virtual machines to host Docker, open the firewall, establishing the network routes, program the monitoring and so on. And the software will be shipped as a docker image, copied by Ansible and launched.
Docker/Containers can simplify your software deployments whether you have a private cloud or regular virtual machines. Simply install the docker system on your virtual machines and change your way to ship your software, past the effort, you won’t regret it.
Day eighty : The good old mama’s Software Factory
The last situation where Docker/Containers is really brilliant is when you use Docker inside your software factory.
Docker can be used in a software factory to make your Software factory evolve from a monolithic all-usage but slow and frustrating software factory to a real platform Software Factory As A Service (If you like the term SFAAS, it’s mine 🙂
The main differences between a Software Factory and a SFAAS are the following : – Product owner, Team manager are creating the new Software Factories for their projects directly through a WebUI by picking the technologies, tools they need. – Developers have the possibility to instantiate new environments to build or test the software without any interaction, paper submission and waiting for a round trip between Earth and Mars. – Integration engineers are providing new tools and environments accessible to the projects, if they wish. – Few interactions are necessary between the infrastructure and system administration teams and the software teams. It’s a win-win solution and the IT bottleneck has been removed.
Docker Software Factory : Marcel Birkner
I strongly recommend that Software factories built on the top of containers like these great initiative projects.
- Software Factory with Docker
- Building a Software Factory with docker, presentation
- Docker infrastructure for Software Factory
Conclusion
If you have read the whole article, I can only say you a Big thank you and I hope you have been able to learn one thing or two. The apparition of containers is really helping developers, ops and I wish that the IT companies fully embrace these technologies to make our profession much funnier and attractive.
The disappointing quest for an Headless CMS in 2017
In 2017, this blog is powered by Hexo.js. However I am looking for a replacement since Hexo.JS is lacking of crucial features.
Introduction
TLDR : HexoJS is too limited, I want online post edition!
I have been recently working to replace the technology powering my blog. A major point is that I am disappointed with its theme. I would like to replace it, with a new technology Vue.JS, for which I have already discussed there.
Since I am replacing the whole front-end, I have been using the great plugin hexo-generator-json. However I still have major issues with my assets (stored with the posts) and it is not really compatible with a CDN solution.
The second feature I am missing, is the possibility to edit my post online. I am an user of Medium and I love the mobile application to create and edits my posts as well watching statistics. A thing I did not think at first, is the impossibility to create new posts with Hexo.JS without an computer. Indeed, to generate your site, you have to generate it, using a full Node.JS environment, commiting, pushing on GITHub your modifications, deploy the docker container and so on. A lot of tasks I have mostly automated but yet, I don’t have a CI Environment available for it.
I did not want to switch back to Drupal and WordPress, equals for me as a bloated solution, slow and hard to tune. I wanted a compromise : why not having a NoSQL Database, a light REST backend, an AdminUI and that’s all. At the beginning of this blog, it was my plan to build this backend, but I decided shortly to concentrate on the content, rather than on the code.
Fortunately, the technologies have evolved and I made a list of Headless CMS / API-First CMD and tested them.
Headless CMS, what is it ?
Headless CMS
I won’t spend too much time in the details, a good description has been done there.
Basically, legacy / traditional CMS are highly coupled solutions where the following components are tied :
- Database : SQL Databases
- Backend : PHP or worse
- Front-End : Templated front end or theme highly coupled with the backend API. Unmodifiable at best, throwable at worst.
- Separated WS / RPC : External service to access the backend data, not used by the front-end.
- Admin UI : Bundled Admin UI.
Usually this kind of CMS are stored in one big block called WordPress, Drupal, Joomla and so on.
The good news is that even these famous solutions are evolving to apply the following modern and well-known principles :
- Decoupled front-end : CMS frontends should be decoupled. The UI will access to the blog data and content using an REST API. UI for Headless CMS are usually using technologies as Angular, React or Vue.JS
- Responsive front-end : Headless CMS enales the possibility to create various UI depending of your devices, smart watch, website, search engine etc.
- NoSQL Database : Handling documents and content is the speciality of NoSQL databases allowing to add your own custom fields, categories and organization.
- Framework : Such Headless CMS should provide libraries or framework to access to the content and handling the security as NPM modules and so on.
- DevOps : Such solution should be dockerized.
My expectations
I am expecting from an Headless CMS to contains :
- a REST backend
- Documented RESTFul Apis
- a database driver compatible NoSQL
- a bundled Admin UI accessing by an API to the REST Backend
- a docker image or docker compose
- possibility to add custom fields
- possibility to handle markdown format for the edition
- Cloud FS Storage for my medias
- Optimized solution : I don’t want a new wordpress installation
- Node.JS solution : I want a lightweight solution
- Self hosted solution : I want to deploy it on Google Cloud.
Results
The list of experiments and my opinion about it.
Directus : No!
Docker-compose was not running (I used this project). The docker instructions worked for me.
I launched it and soon enough I received a lot of technical alerts wasting my pleasure of a fresh installation.
Directus / Error message
My last blocking point, and the reason I have rejected : I did not find any way to create a content category (called table) in the admin UI. Seems to manipulate the SQL Database to create them : no thanks (rant here).
GetMesh : Meh
Uh Uh, a Java solution to power a small blog : no thanks.
Drupal and WordPress : Hydra CMS
Too big, too well-known. The REST API is for sure the next security hole of these solutions.
But the reason of my reject, the UI cannot be separated from the backend!! And why would I like a UI embedded in my backend when I want to create a SPA WebSite ?
I will use them when they will have deleted their UI from the installer.
I suggest to call them HydraCMS.
GraphCMS : Hipster$$CMS
GraphCMS
Looks Great but I want my own self-hosted solution and don’t want to pay for that.
Site here
Ghost : GirlyCMS
Honestly, I had a crush with Ghost. Sexy, a great installer, a great documentation, everything to tempt me like an attractive woman.
The problem is that Ghost has almost everything to charm me but he has an embedded UI!!!
I don’t want an UI, I want to build mine 🙁
Apart from that point, GhostCMS is really great.
Ghost CMS
It even has an Slack integration and loves Markdown!!
Cockpit : Blind CMS
Cockpit CMS
Listed in the Awesome CMS List, Cockpit CMS is a rather small solution.
The good points are :
** Docker is working fine. ** The concepts and architecture are OK. ** Nice AdminUI, I really appreciated the way to create my collections
But what really disappointed me was :
** No documentation (REST and so on). For an developer it’s unusable ** PHP : There is no documentation and the REST API is coded in PHP.. Meh ** Lonesome developer yes he is brave, we should encourage him, but he is freaking alone.
In summary, I think that this project goes in the right direction but took some tough and spiky path. PHP is clearly not the appropriate language for such solution. Compared to an Express server, the amount of work to be delivered is too high. It really needs more contributors (actives) to create a good solution and fill the big documentation blackhole. I cannot help since I don’t want to code again in PHP but the solution could be great.
Site is here
KeyStoneJS
Well at the first glance, I rejected, could not find any Docker image. Or the few ones were not working. But my first attempt was dumb. KeystoneJS is not an Headless CMS by itself it’s rather an implementation of a CMS, fully customizable to create your own blog!
Powered by Express and Node.JS, two technologies I am particularly fond of!
The site is there
The positive sides of KeyNodeJS :
- A slick project creator using Yeoman!
- Modern technologies, I think the best to create a CMS
- The bundle is containing what I am expecting (AdminUI, REST Backend, NoSQL Database(mongoDB))
- Fully customizable collections and so on
The negative points are :
- Maybe too much code to begin with
- What is the maturity of the base implementation ?
- How much effort requested to build its own website ?
- I think I did not find yet a NPM module to build a REST Client
Conclusion
I have rejected most of these solutions.
- I tried two times an installation and to migrate my data in DirectUS but I gave up. I don’t believe in the concepts.
- The lack of API Documentation in Cockpit (HTML or a la Swagger) is blocking my attempt to use it and migrate my data. The fact that the solution is developed in PHP is blocking my wish to support them. And I don’t like much PHP REST Backend to be honest.
- I really love Ghost but I don’t want their UI, I want mine. Otherwise I would have use it.
- I tried to use Drupal and WordPress, but the requested system resources + the fact I cannot disable the UI are a big NO for me.
The consequence is that I am using KeystoneJS and I hope I won’t have too much work to power a new version of my blog.
Stay tune!
References :
**
Disruption in Software Quality Assessment ?
As many other markets, the SQA/ALM Market soon will meet #disruption. Domains like machine learning, deep learning and cloud computing will force it to evolve in the next few years. This article is presenting some predictions about the future of the quality tools.
Disruption in Software Quality Assessment
Disclaimer I am not a native english speaker and I am perfecting my english skills by writing these articles. If this topic interests you, please comment below or share the article to your friends. And every syntax, grammar mistakes will be fixed under your wise comments.
A new generation of Software quality tools is going to emerge. Machine Learning, Deep Learning, DevOps, Continuous Delivery, Continuous Integration, Cloud Computing, all these movements are influencing the SQA/ALM Software Editors. It has never before been so easy and cheap to produce a new static analysis tool to measure some aspects of a software. The Opensource movement and the market evolution are the direct contributors to this state. Made famous under the name of “linters”, well-known and unknown developers are creating the tools required to their activities. And the Software editors are faced to the dilemma : “Should I continue to build my own tools ? What should be my behaviour confronted with this plethoria of scanners ?”.
Until recently, Software developers were depending of the highly-specialized skills from the Quality Software Editors to detect, analyze and fix the bugs inside their softwares. And it is a big source of frustration. From both sides. Developers are usually complaining that the rules do not reflect their real needs or the complexity of their softwares. “Quality tools do not detect real problems or too late or under a trillion of false positives”. Software Editors are providing to the hungry population rule sets, standards to satisfy the crowd. A crowd much much bigger than their own forces.
I am predicting that the disruption may be coming from these directions :
- From the open-source : soon or later, the basic needs of developers will be fulfilled by the open-source offer. Tools like PMD, Findbugs, and so on have inspired a whole generation of developers. The young developers through the Angular 2, ReactJS, Go are already educated to the benefits of Quality tools. And they are heavily relying on linters well-integrated in their CI or in their IDE (Atom, Code). Twitter, Facebook are continuously producing and releasing in opensource new tools to help the developer community. The recent examples of Flow or PrePack are helping a lot developers to increase the quality of their products.
- From the digital technologies. The increasing level of maturity of the machine learning and deep-learning technologies should bring us shortly new kind of tools to predict bugs, predict code defects and usual developer decisions. I believe that the scientific researches from Microsoft and Google will contribute indirectly to the Software Quality tool market. This topic is unsurprisingly very discussed (here).
- From the software development process transformation : Movements like Agile, DevOps, Continuous Integration and Deployment, ChatBots are deeply changing the way developers are collaborating. Several aspects are changing : communication (Slack, Hipchat), software building (Jenkins, Travis CI, Microsoft TSF & Azure), software deployment (containers, PAAS, Amazon AWS)… The way a product is conceived, built and deployed requires to track and measure several quality aspects. The integration effort to produce these metrics and KPI’s is tremendous and have to be adapted to each organization. Would the developers be enough satisfied with code quality or will they require higher levels metrics extracted from their development process.
Conclusion
Who will be the future leaders in the ALM market ? Who will be the fastest to adapt to the current technology and data disruption ? Do you have some tools that could match these descriptions ?
If that article has been useful or interesting, stay connected, I will produce new articles on that subject.
One of my future article will present Codacy, an emerging code quality platform. This platform offer to ease the quality control as soon as possible in your development process to detect the bugs early and surely. I will compare this solution with the famous market leader SonarQube.
Codacy : an easy-to use code quality review solution
Some time ago, I organized a talk on “Code Quality and why developers should care about” for my company. In that presentation, I demonstrate some tools, unfamiliar for the audience : young developers and battle-hardened IT developers. One among these tools has brought to me a lot of questions : Codacy. I will present its features and perform a comparison between Codacy and SonarQube from SonarSource.
Recent Comments