Sunday 10 July 2016

How to build Kibana from Source!!

Dear All,

In this blog post, I am going to summarize how I built Kibana - open source, browser-based analytics dashboard, from source code.

Environment: Linux RHEL from AWS

Version: 4.4 (At the time of writing this article, master version of Kibana is 5.0. But my requirement is to build it in 4.4 version.)

Visit the github page of Kibana here and select the branch version of 4.4. You will be landing here.

This page provides a detailed information about building Kibana, but I had to do few more tweaks - I will be explaining them in this blog.

Steps:

1. Clone the version 4.4 from Kibana source.
git clone -b 4.4 https://github.com/elastic/kibana.git
cd kibana

2. Check the version of npm from the file .node-version and install the respective npm using below command.
nvm install "$(cat .node-version)"

3. Install the dependencies and start the Kibana and Elasticsearch using below commands:

(Please note that below commands would require lot of dependencies to be installed in linux machine. Please use 'Yum' installer for this purpose.)

npm install
npm run elasticsearch
npm start

Hope everything is fine till this point. In the coming steps, I will be explaining about the build process.

4. Navigate to the OS packages section of the page and execute below commands:
gem install pleaserun
gem install fpm
npm run build:ospackages

5. When I ran the above command, system displayed an error as below related to some path issue:

Invalid package configuration: Cannot package the path '/home/ec2-user/kibana_src1/kibana_0711/build/services/systemd/lib', does it exist? {:level=>:error}

Upon googling, I found that this issue is related to pleaserun package update as described here:

https://github.com/elastic/kibana/issues/7265

Initial suggestion was to downgrade pleaserun package, but later it also did not work - as fpm package internally uses the latest version of pleaserun.

So, I had to manually update the 'tasks/build/osPackages.js' file as per the fix provided here:

https://github.com/elastic/kibana/pull/7276/files

6. Open the file 'tasks/build/osPackages.js' using vi editor and replace the below code from:

      const systemd = servicesByName.systemd.outputDir + '/lib/=/lib/';

      const systemd = servicesByName.systemd.outputDir + '/etc/=/etc/';

7. Above change would resolve the error associated with file path.

8. But upon rebuilding, I was getting another error related to 'shasum' package.

What I did is, I copied the content of 'shasum.js' file from master branch and pasted it in 'tasks/build/shasum.js'. Also, have modified below line from

var cmd = /^win/.test(platform) ? 'sha1sum ' : 'shasum ';

to

var cmd = /^win/.test(platform) ? 'sha1sum ' : 'sha256sum ';

This is how updated shasum.js file was looking like:

var { promisify } = require('bluebird');
var readdir = promisify(require('fs').readdir);
var exec = promisify(require('child_process').exec);
var platform = require('os').platform();
var cmd = /^win/.test(platform) ? 'sha1sum ' : 'sha256sum ';

module.exports = function (grunt) {
  grunt.registerTask('_build:shasums', function () {
    var targetDir = grunt.config.get('target');

    // for when shasums is run but archives and ospackages was not
    grunt.file.mkdir(targetDir);

    readdir(targetDir)
    .map(function (archive) {
      // only sha the archives and packages
      if (!archive.match(/\.zip$|\.tar.gz$|\.deb$|\.rpm$/)) return;

      return exec(cmd + archive + ' > ' + archive + '.sha1.txt', {
        cwd: targetDir
      });
    })
    .nodeify(this.async());
  });

};

9. After making above modification, build ran like a charm and I have a working copy of Kibana available in various os under 'build' folder.

Please let me know if you need any additional information.


1 comment:

  1. hey.I don't know if you still are.but the shasum work truly.
    thanks very much

    ReplyDelete