When I was writing the code for qlBuilder reload command I knew I'll have to come back to it, at some point, and refactor the part that is displaying the reload progress. The plan was to separate this functionality as a separate module or even a package so it can be used elsewhere as well.

And that time has finally came. But instead of a separate package this functionality is now part of enigma-mixin package.

The usage of mGetReloadProgress mixin functionality is a bit niche but it can be used to get the reload progress while reloading an app via enigma.js (same as what Qlik is displaying when reloading an app).

With vanilla enigma.js we have to use GetProgress() API method. But working with it can be a bit tricky. Or at least not trivial.

The main difference between this and the majority of the other API methods is that reloading an app is a long running process and we'll have to call GetProgress() periodically (every X ms) while the app is reloading in order to get the current progress.

The principle here is to not await for the DoReload() (or DoReloadEx()) method but to wrap it in a Promise and inside it to start the reload. After the reload promise is started we'll start pooling every X ms to get the reload progress. Once the doReload() is complete we'll resolve the main promise and return the doReload() response.

const doc = await global.openDoc("some-app-id");

const reloadApp = await new Promise(function (resolve, reject) {
    // start the app reload
    doc.doReload().then((r) => {
        // once the app is reloaded resolve the main promise
        resolve(r);
    });

    // while the app is reloading get the reload progress
    // every 200ms and do something with the response data
    setInterval(function () {
        global.getProgress().then(function (msg) {
            //do something with the message here
        })
    }, 200)

});

// continue here after the app reload is complete

The above snippet gives us an idea how to work with getProgress() method. The next (main?) issue will be to parse the response data of getProgress(). And that can also be challenging due to the nature of the messages/progress.

As you can see getting the reload progress is not as simple as it sounds.

To simplify the process of getting the reload progress I've prepared an enigma.js mixin. More about mixin can be found here and here

By "simplify" I mean that you won't have to deal with getProgress() response data and the hassle that this can bring. I can talk a bit more about the response structure but that can be ... boring 😄(Reach out if you want to know more)

The pseudo code below demonstrates how to import and use the mixin into enigma.js session and how to use it:

import { globalMixin } from "enigma-mixin";

const session = enigma.create({
  schema,
  mixins: globalMixin,
  url: `ws://my-qlik-sense/app/engineData`,
  createSocket: (url) => new WebSocket(url),
});

// open the web socket connection
const global = await session.open();
// open the required app
const doc = await global.openDoc("some-app-id");

// init the get progress mixin
const reloadProgress = global.mGetReloadProgress();

// once the mixin is initialized we can
// prepare the emitter
reloadProgress.emitter.on("progress", (msg) => {
    // here we will receive the reload messages
    console.log(msg);
});

const reloadApp = await new Promise(function (resolve, reject) {
    // or doc.doReloadEx
    doc.doReload().then((r) => {
        // give it another few ms to make sure we have all the messages
        setTimeout(function () {
            // stop the mixin from pooling for new messages.
            // The app reload is already complete at this point
            reloadProgress.stop();
            // resolve the main promise
            resolve(r);
        }, 300);
    });

    // once the reload is started we'll start the pooling
    reloadProgress.start();
});

The parsed and formatted reload messages will be available inside the reloadProgress.emitter.on("progress")... function. At the end the messages here will (should) be identical to what is displayed in Qlik.

Follow the link below for the source code and more details about this specific mixin or if you want to know more about the other available mixins. The mGetReloadProgress mixin have a few (optional) parameters that can be set. Like how often to pool for progress and to include/exclude all transient messages.

GitHub - countnazgul/enigma-mixin: Set of custom enigma.js mixins

Set of custom enigma.js mixins. Contribute to countnazgul/enigma-mixin development by creating an account on GitHub.

GitHubcountnazgul

P.S. The same setup can be used with doSave() method as well. Although it seems that Qlik is not sending any data in the response. Not sure if this is helpful in some way ... just saying 😉

Stefan

While working with Qlik we can encouter errors or messages that can be a bit cryptic and contain just an error/message code.

Very often these codes are followed by a short description. But sometimes either the description is missing or its ... to short :)

Finally found some free time to compile a list of the error/messages codes, their short and long description.

My data source is a few years old so most likely is not complete. But still there are 300+ codes there.

GitHub - countnazgul/qlik-internal-codes: List of hard to find Qlik internal codes and their description

List of hard to find Qlik internal codes and their description - countnazgul/qlik-internal-codes

countnazgulGitHub

If you happen to know more of these, please feel free to open PR in the repo.

Stefan

The original announcement can be found here

Automatiqal CLI is officially entering its Beta phase!

You might not be aware but Automatiqal CLI allows Qlik Sense (QSEoW) deployments/administration tasks to be described in YAML/JSON files.

Automatiqal CLI supports:

• multiple authentication methods
• multiple ways to define variables
• define onError blocks
• and a lot more!

Please have a look at the introduction video below. More videos will be added soon!

Any feedback is greatly appreciated!

Stefan

In one of the previous post we saw how to use web ticket to establish connection with Qlik Sense Engine. And probably for most cases the approach there is enough.

But what if our web app have a requirement to communicate with other Qlik service(s) (in addition to the Engine)? For example, we want to list the reload tasks to which the user have access (this information is provided by Qlik Repository Service REST API)?

The problem with the web ticket is that once it is consumed it can't be used again. And if we use the ticket to open web socket Engine connection then we can't use the same ticket to get the list of the reload tasks. We'll end up with situation where the Engine connection is authenticated and established but upon requesting the list of reload tasks Qlik will response with Unauthenticated.

To overcome this we can "exchange" the web ticket for a Qlik session. The exchange will be in a form of a session cookie. We'll have the session cookie set in the browser and then our web app can freely open Engine connection (without web ticket) and communicate with other Qlik Sense services. In similar way the Hub is doing it.

Pre-requisites (QMC)

Couple of settings have to be set in the Virtual Proxy before we can start (if they are not set already):

• White list - add the web app url to the host white list

• SameSite attributes - make sure that the SameSite attributes are set to None (at least)

Code workflow

The code workflow is:

• get web ticket from Qlik (in the same manner as in the previous post)
• use the ticket to retrieve the content of any static Qlik resource via HTTP request. In this example I'm going to use /resources/autogenerated/qlik-styles.css but it can be any other static file that requires authentication in order to be viewed/downloaded
• since we are using the ticket in HTTP request the response from Qlik will be the content of the requested resource + session cookie
• once session cookie is set (the browser is taking care of this) we can continue and connect to the Engine and "ask" Qlik for different data (in our case get list of reload tasks)

Code

• retrieve the static file content

export const load = async ({ url, params }) => {
  // look for "qlikTicket" query parameter
  const qlikTicket = url.searchParams.get("qlikTicket");

  // if "qlikTicket" is present
  // use the ticket to retrieve the content of "qlik-styles.css"
  // if all is ok Qlik will send the session cookie
  // and the browser will set it
  if (qlikTicket) {
   const qlikStylesURL = `https://${qsHost}/resources/autogenerated/qlik-styles.css?QlikTicket=${qlikTicket}`;
 
  const qlikStylesResponse = await fetch(qlikStylesURL, {
    credentials: "include",
   });
 }

  return true;
};

• variables

const qsHost = "my-sense-instance.com";
const reloadURI = `http://localhost:3000`;
const xrfkey = "1234567890123456";
const qsReloadTasksURL = `https://${qsHost}/qrs/reloadtask?Xrfkey=${xrfkey}`;
 
let qlikApps = [];
let qlikReloadTasks = [];

• Notification events

// redirect to the login page if the Engine connection response indicates that we have to authenticate
session.on("notification:OnAuthenticationInformation", (data) => {
 if (data.loginUri) goto(data.loginUri);
});

// once the Engine connection is established, get the list of the reload tasks (in the logged in user context)
session.on("notification:OnConnected", async () => {
 qlikReloadTasks = await fetch(qsReloadTasksURL, {
   credentials: "include",
   headers: {
     "X-Qlik-Xrfkey": `${xrfkey}`,
   },
 }).then((res) => res.json());
});

• HTML structure

<main>
  <div>
    <h1>Qlik apps:</h1>

    {#each qlikApps as qlikApp}
      <div title={qlikApp.qDocId}>{qlikApp.qDocName}</div>
    {/each}
  </div>

  <div>
    <h1>Qlik Reload Tasks:</h1>

    {#each qlikReloadTasks as reloadTask}
      <div title={reloadTask.id}>{reloadTask.name}</div>
    {/each}
  </div>
</main>

Result

Once the whole process is complete we should see something like:

And if we inspect the network request will see:

• the qlik-styles.css request is with attached QlikTicket the response is 200 OK and the response headers are containing the session cookie

• we can see the cookie being set in the browser

• both the Engine connection and Repository API request are made without any ticket added to the urls

Complete code

The full code is available at the repo below. Follow the readme there to install and run it.

GitHub - countnazgul/qseow-svelte-authentication-session

Contribute to countnazgul/qseow-svelte-authentication-session development by creating an account on GitHub.

GitHubcountnazgul

Hope you liked it!

Stefan

The initial idea was simple ... on paper. Spend few days just to prove that its possible. And it was doable. But to achieve it I had to build solid foundation.

I'm not going to say what was the initial idea ... yet. But will share where it's "home" will be and will release all the components of the "foundation".

Since there will be multiple components I've decided that it's a good idea to separate them in their own space.

So let me introduce you to Informatiqal

Informatiqal has it's own website and it's own GitHub organization:

I N F O R M A T I Q A L

Ask us about Qlik Sense API :)

I N F O R M A T I Q A L

Informatiqal

Ask us about Qlik Sense API :). Informatiqal has 5 repositories available. Follow their code on GitHub.

GitHub

Code for a few mode modules is publicly available but want to meet you with the "base of the base". The others are still WIP.

Qlik REST API

This a "generic" package. It simplifies the communication with various Qlik Sense REST API endpoints (Repository, Proxy, SaaS etc.) but still we have to know what actual endpoint have to be called and what data has to be passed.

But on top of it few more packages are/will be created which are going to make the development process more pleasant.

qlik-rest-api can be used from within both NodeJS and browser(s). The example below shows how certificate authentication can be used (from NodeJS) to communicate with Qlik Repository REST API.

import fs from "fs";
import https from "https";

// read the certificates
const crt = fs.readFileSync("path/to/certificate.pem");
const key = fs.readFileSync("path/to/certificate_key.pem");

// init https agent
const httpsAgent = new https.Agent({
  rejectUnauthorized: false,
  cert: crt,
  key: key,
});

// create the config object by passing the created httpAgent
const config = {
  host: "my-qlik-sense-instance",
  port: 4242, // optional. default is 4242
  httpsAgent: httpsAgent,
  authentication: {
    user_dir: "SOME_DIR",
    user_name: "SOME_USER",
  },
};

// initialize the repo client by passing the conifg object
const repoClient = new QlikRepositoryClient(config);

// invoke Repository API "about" endpoint
const result = await repoClient.Get("about");

And the result variable will be in format:

And the data property will hold the response data:

Feel free to check the package repository page. Don't forget to check the Wiki section for more details and examples.

GitHub - Informatiqal/qlik-rest-api: Interact with multiple Qlik Sense REST APIs from a single package

Interact with multiple Qlik Sense REST APIs from a single package - GitHub - Informatiqal/qlik-rest-api: Interact with multiple Qlik Sense REST APIs from a single package

GitHubInformatiqal

Dev documentation of all exposed methods, types and interfaces can be found at:

P.S. Please feel free to open issue for any question or bug you encounter :)

Hope you liked it!

Stefan

This post is long overdue but finally found the time to complete it.

Here we'll have a look how to establish connection (via enigma.js) to Qlik Sense Enterprise on Windows (QSEoW) when hosting an web app outside of Qlik (not as an extension). And more specifically we'll have a look at what is the authentication work flow.

The (very simple) web app will just list the apps that are available for the logged in user. I'll be using Svelte as web framework (more specifically SvelteKit) but the general work flow should work in the same way with other frameworks as well.

Let's begin!

The problem

When our web app is hosted by Qlik (as an extension) we can only access it through Qlik's extensions url (like: https://my-sense-instance.com/extensions/MyMashup/index.html) It's not a very nice URL. And every time when new version of our web app is ready we'll have to remove the old extension and re-upload the new version. (of course this can be automated)

The benefit of having the web app hosted by Qlik is that in order to access it you'll have to be authenticated. If not - Qlik will handle the redirection.

In our case we will redirect the user to Qlik's login page, receive web ticket and use the ticket to establish connection with the engine.

Scenario

The scenario is very simple:

• using enigma.js to establish session with Qlik Engine.
• once the session is esablished get list of all user's apps
• display the apps list on the page

Session events

Crucial part of the authentication work flow is enigma's capability to "listen" for specific events (check out the Session API part of the documentation)

Enigma can "listen" for various events (like session closed, session suspended, all the traffic send/received to/from the Engine or any named notification).

In our case we'll listen for OnAuthenticationInformation event. This event can tell us if we must authenticate and if we must it will provide us with the URL to the Qlik's login page. Once authenticated there Qlik will re-direct us back to the web app but with the Qlik ticket in the url. The ticket then can be used when establishing connection with enigma

General workflow

1. When the user opens the web page we'll try and connect to QSEoW via Web Sockets (await session.open())
2. If the users is not authenticated Qlik will "raise" OnAuthenticationInformation notification event. The event contains simple JSON object:

{
  mustAuthenticate: true,
  loginUri: "https://..."
}

3. In our code we are listening for OnAuthenticationInformation events and when receive one (and mustAuthenticate: true) we are going to redirecting the user to the loginUri value from the event
4. loginUri is the main Qlik login page (not part of our web app!). The user enters their credentials there
5. If the credentials are ok then Qlik redirect back to our web app page but with qlikTicket added to the url as a parameter
6. Again we send session.open() but this time we'll provide the qlikTicket
7. Qlik replies with SESSION_CREATED
8. And at this point we are all set and can continue communicating with the Engine. All requests will be made in the scope of the logged user

The image below illustrate what is the general work flow to establish session and authenticate

Code

The first bit of code is to handle the URL parameter. If the URL contains `qlikTicket` we'll pass the value as property. When the page is loaded for the first time the value of `qlikTicket` will be empty.

<script context="module">
  export const load = async ({ url }) => {
    let qlikTicket = url.searchParams.get("qlikTicket");
    return { props: { qlikTicket } };
  };
</script>

Importing enigma and `goto` from Svelte navigation. `goto` will be use to re-direct to the login url.

<script>
  import { goto } from "$app/navigation";
  import enigma from "enigma.js";
  import schema from "enigma.js/schemas/12.67.2.json";

Define Qlik host and URL for the web app. The web app location can be found  programmatically of course.

  const qsHost = "wss://my-sense-instance.com";
  const reloadURI = `http://localhost:3000`;

Define few variables:

• qlikTicket - will hold the Qlik ticket (when we have it)
• qlikApps - will hold the list of the Qlik apps
• qlikTicketString - the ticket have to be passed as URL parameter itself. This variable will contain this string

export let qlikTicket;
let qlikApps = [];
let qlikTicketString = qlikTicket ? `&QlikTicket=${qlikTicket}` : "";

Generate the session object. The interesting part here is the url. The url is containing three "elements":

• ${qsHost}/app/engineData - setting the Qlik host as usual
• reloadURI=${encodeURIComponent(reloadURI) - this part tells Qlik to return to this location once the user is successfully authenticated (in our case this will be http://localhost:3000)
• ${qlikTicketString} - when we have the ticket it will be passed here

const session = enigma.create({
    schema,
    url: `${qsHost}/app/engineData?reloadURI=${encodeURIComponent(
      reloadURI
    )}${qlikTicketString}`,
    createSocket: (url) => new WebSocket(url),
  });

The session event part. If notification of type `OnAuthenticationInformation` is received and the event data contains `loginUri` then re-direct the page to this url. Once the user authenticate there Qlik will return us to our web page but with the ticket appended at the end. The url will looks like:

http://localhost:3000/?qlikTicket=YMnqm8WvW8AXSA99

  session.on("notification:OnAuthenticationInformation", (data) => {
    if (data.loginUri) goto(data.loginUri);
  });

The main logic. Once the session is successfully established - get the list of the apps and populate `qlikApps` variable with it.

  (async function () {
      let global = await session.open();
      qlikApps = await global.getDocList();
      console.log(qlikApps);
      await session.close();
  })();
</script>

The HTML part is very basic:

<main>
  <h1>Qlik apps:</h1>

  {#each qlikApps as qlikApp}
    <div>{qlikApp.qDocName}</div>
  {/each}
</main>

And the result:

Once the local development is complete the bundle can be hosted anywhere

Complete code

The full code is available at the repo below. Follow the readme there to install and run it.

GitHub - countnazgul/qseow-svelte-authentication

Contribute to countnazgul/qseow-svelte-authentication development by creating an account on GitHub.

GitHubcountnazgul

Hope you liked it!

Administrating Qlik Sense sometimes means spending a lot of the time in QMC.

And sometimes I need to export some of the tables there just to keep the info somewhere else while I'm on a different screen in QMC.

Finally sat down to solve this specific issue - Chrome/Edge extension that can export QMC tables to CSV file or copy the content to the clipboard.

UI

The extension layout is "separated" into two areas:

• Copy - exports the table and paste it into the clipboard. The only option here is the delimiter - comma (default) or tabs (will make the life easier if we are pasting in Excel
• Export - exports the table and downloads it as CSV file. The file name can be specified (without the .csv) or it will be auto generated (in uuid format)

Browser permissions

The extension require the following browser permissions:

• activeTab - access to the current active tab
• hosts - access to the host matching the patterns https://*/qmc/* and https://*/*/qmc/*. This should filter only the host with QMC (with or without virtual proxy). If the extension is started on a host which is not matching the patterns (for example https://stackoverflow.com/) the following error message will be shown:

If the extension is started on correct host but there are no tables found then the following message will be shown:

Installation

The extension is available for Chrome, Edge (Chromium) and Firefox and can be downloaded from Google and Microsoft extension stores:

• Chrome Web Store
• Edge Add-ons Store
• Firefox Add-ons store

Hope you liked it!
Stefan

Another interesting feature of enigma.js are interceptors. Interceptors are similar to the mixins functions but they are executed on lower level - request/response level.

Request interceptors are invoked right before the request is send to the Engine and response interceptors are invoked right after the response from the Engine is received and before the data is send to the calling function.

You can think of the interceptors as "enigma.js middleware".

There are couple of functions available for request and response:

• Request
  • onFulfiled - invoked just before the request is going to be send to the Engine so you can alter the request is needed
• Response
  • onFulfiled - invoked when the response from the engine is ok (no errors)
  • onRejected - invoked when the Engine returns an error so you can handle the errors on enigma.js level

The code below shows how to construct request interceptor:

const enigma = require('enigma.js');
const schema = require('enigma.js/schemas/12.20.0.json');
const WebSocket = require('ws');

const session = enigma.create({
  schema,
  url: 'ws://localhost:9076/app/engineData',
  createSocket: (url) => new WebSocket(url),
  requestInterceptors: [{
    onFulfilled: function myHandler(sessionReference, request) {
      // do somthing with the request data here
      // OR return the original request 
      return request
    },
  }],
});

Example

A simple example of response interceptor can be - change the return data format of a getLayout() method which gets the current selections data.

This interceptor will "listen" for getLayout requests and if the response is for a specific object type it will "flatten" the returned data and return it in the following format:

[
    ...
    {"field": "name-of-the-field", "value": "selected-value-1"},
    {"field": "name-of-the-field", "value": "selected-value-2"},
    {"field": "some-other-field", "value": "selected-value-1"}
    ...
]

• First lets prepare the session object and define our interceptor there:

const session = enigma.create({
    schema,
    url: 'ws://localhost:9076/app/engineData',
    createSocket: (url) => new WebSocket(url),
    responseInterceptors: [{
        onFulfilled: function toggleDelta(session, request, response) {
          // check if the request method is getLayout and 
          // the response is for object with type current-selections-modified
            if (
                request.method === 'GetLayout' && 
                response.qInfo.qType == 'current-selections-modified'
            ) {
                // if the above is valid then "flatten" the response
                return response.qSelectionObject.qSelections.map((s) => {
                    return s.qSelectedFieldSelectionInfo.map((f) => {
                        return {
                            field: s.qField,
                            value: f.qName
                        }
                    })
                }).flat()
            }

            // for every other response return the original response 
            return response;
        },
    }],
});

• Let's select some values first:

    let field = await qDoc.getField('Product Sub Group Desc')
    let selectValues = await field.selectValues([
        { qText: 'Ice Cream' },
        { qText: 'Juice' },
        { qText: 'Chips' }
    ])

• Once we have the interceptor code ready and some selections are made then we can prepare the properties of the object that will hold the selections:

    let selectionObjectProps = {
        "qInfo": {
            "qId": "",
            "qType": "current-selections-modified"
        },
        "qSelectionObjectDef": {}
    }

Check out the type of our object - current-selections-modified. Our interceptor will change the data output only if the response is for this object type. This way we will not "disrupt" the data for the other object types.

• once we have the properties set, we continue our enigma calls as usual: create session object and get its layout:

let sessionObj = await qDoc.createSessionObject(selectionObjectProps);
let sessionObjLayout = await sessionObj.getLayout(); // out interceptor will be invoked here
console.log(sessionObjLayout)

The result of the last row will print the following "flatten" array:

[
  { field: 'Product Sub Group Desc', value: 'Juice' },
  { field: 'Product Sub Group Desc', value: 'Ice Cream' },
  { field: 'Product Sub Group Desc', value: 'Chips' }
]

It's probably not the most practical example but it will give you an idea what enigma.js interceptors are. There are couple of more examples in enigma.js repository itself:

qlik-oss/enigma.js

JavaScript library for consuming Qlik’s Associative Engine. - qlik-oss/enigma.js

GitHubqlik-oss

Conclusion

The same interceptor can be written as mixin as well and instead intercepting the response just call the mixin which will return the same result. But if you want low level / fine grain control over the request/response traffic then interceptors are the way.

Code

Check out the repository below for the full example code

countnazgul/enigma-js-interceptors

Small script to demonstrate the usage of enigma.js interceptors - countnazgul/enigma-js-interceptors

GitHubcountnazgul

Hope you liked it!

Stefan

In the previous post we saw what are enigma.js mixins. I've started an repo some time ago (and npm package) where I've been adding mixins that I've found useful (or mixins that I've found boring to write all the time). And want to introduce you to this list.

At the moment the repo contains only mixins that are relevant to application/document.

(to distinct the mixins methods from the default enigma.js ones all mixin methods, from this package, are having prefix m)

The mixins are "split" into groups:

• Selections

  • mSelectionsAll - returns the "native" reponse from Qlik ragarding the current selections

  • mSelectionsFields - an array of only the field names, having selections in it

  • mSelectionsSimple - without any parameters returns a flat array with all selections

    [
      {field: "Field 1": value: 'Value 1'},
      {field: "Field 1": value: 'Value 2'},
      {field: "Field 2": value: 'Value 10'},
      ...
    ]
    

    if groupByField parameter (optional and default is false) is passed then the result will be grouped by a field name:

    [
      {field: "Field 1": values: [...]},
      {field: "Field 2": values: [...]},
      ...
    ]
    

  • mSelectInField - provide an field name and array of values to be selected

• Tables and fields
  • mGetTablesAndFields - returns an array with {table: "data-table-name", field: "field-name"} values
  • mGetTables - an array with all data tables
  • mGetFields - an array with all fields
  • mCreateSessionListbox - creates a listbox session object for the provided field
• Variables
  • mVariableGetAll - array with all variables
  • mVariableUpdateById - updates the definition of an existing variable by providing the variable ID
  • mVariableUpdateByName - updates the definition of an existing variable by providing the variable name
  • mVariableCreate - creates a new variable from the provided name, comment (optional) and definition
• Extensions
  • mExtensionObjectsAll - returns an array with all extension objects in the current app. Returned fields: appId, appName, objId, objType, extName, extVersion, extVisible, extIsBundle, extIsLibrary, qProps
• Unbuild
  • mUnbuild - extracts all parts of an Qlik Sense app (apart from the data itself) into JSON object
• Build
  • mBuild - the opposite of un-building - provide a json object with the app components and this mixin will update (or create) all the objects in the target app

If you want to use the build and unbuild commands outside JavaScript then I highly recommend using Corectl tool

qlik-oss/corectl

corectl is a command line tool to perform reloads, fetch metadata and evaluate expressions in Qlik Core apps. - qlik-oss/corectl

GitHubqlik-oss

Installation

The installation is very simple:

npm install --save enigma-mixins

Usage

(make sure that enigma.js is already installed)

import enigma from "enigma.js";
import schema from "enigma.js/schemas/12.20.0.json";
import enigmaMixins from "enigma-mixins";

// Create enigma session as usual but include the mixin property
const session = enigma.create({
  schema,
  mixins: enigmaMixins, // this is where the mixinx are added
  url: "ws://localhost:4848/app/engineData",
  createSocket: (url) => new WebSocket(url),
});

Once the session is established and the document is open, then you'll have all the mixins available:

Contribution

Feel free to open an PR with your mixins :)

Repository and npm

countnazgul/enigma-mixin

Set of Enigma mixin. Contribute to countnazgul/enigma-mixin development by creating an account on GitHub.

GitHubcountnazgul

enigma-mixin

Set of Qlik Sense enigma.js mixin

npm

Hope you liked it!

Stefan

You know it already but enigma.js is the JavaScript wrapper around the Qlik's Engine API.

If you are building a mashup/web app or some sort of automation that require Engine communication you'll have to use Engine API and here enigma.js is your friend.

Just to mention that JavaScript is not the only "flavor" of Enigma. You can find enigma for Go and .NET:

qlik-oss/enigma-go

Go library for consuming Qlik’s Associative Engine. - qlik-oss/enigma-go

GitHubqlik-oss

q2g/enigma.net

.NET library for consuming Qlik’s Associative Engine. - q2g/enigma.net

GitHubq2g

But in this post I want to introduce you to one very cool (but not very popular) functionality that Enigma provides - mixins.

Mixins allow us to extend, with additional methods, or override existing methods. Mixins are bound to at least one type (type being - Doc, GenericObject, GenericBookmark etc.). The official documentation can be found in the official enigma.js repo.

Let's write some code (basics)

Essentially each mixin is a JS object with few properties - init, types and extends and/or overrides.

A very simple mixin will be:

const docMixin = {
    types: ['Doc'],
    init(args) {
        console.log(`My mixin is being initialized`);        
    },
    extend: {
        myMixin() {
            console.log('My mixin was called!');
        },

    }
}

From the snipped above, we can see:

• this snipped is available for docs (apps)
• when the mixin is initialised a message will be printed in the console
• when its called the mixin will only print a message

To use the mixin we have to add this object (or objects) to the Enigma session:

const session = enigma.create({
    schema,
    mixins: [docMixin],
    url: 'ws://localhost:9076/app/engineData',
    createSocket: (url) => new WebSocket(url),
});

Once we have the session established then we can open an app and use our mixin:

let global = await session.open()
let doc = await global.openDoc('/data/Consumer Sales.qvf')
doc.myMixin()

In our case the mixin is pretty general (just printing a message) and we might want to use it for other types and not just in the apps context. To add it to more types just edit the type array:

...
types: ['Doc', 'GenericObject', 'GenericBookmark']
...

A bit more advanced example

Getting the layout of an object is a two step process:

let qObject = await doc.getObject('some-object-id-here');
let qLayout = await qObject.getLayout();

What about if we write an mixin that is doing these two steps for us so we can have a single command to get the layout? An example mixin can look like this:

const docMixin = {
    types: ['Doc'],
    init(args) { },
    extend: {
        getLayoutMixin(objectId) {
            let qObject = await doc.getObject(objectId);
            let qLayout = await qObject.getLayout();

            return [ qObject, qLayout ]
        }
    }
}

const session = enigma.create({
    schema,
    mixins: [docMixin],
    url: 'ws://localhost:9076/app/engineData',
    createSocket: (url) => new WebSocket(url),
});

let global = await session.open()
let doc = await global.openDoc('/data/Consumer Sales.qvf')

let [myObject, myObjectLayout] = await doc.getLayoutMixin('some-object-id');

Conclusion

Personally I found the mixins to be very cool and handy feature. But also I found them very underused (including myself here as well).

I've been setting my own collection of useful mixins in my free time and the next post will be about this collection but you can have a look in the repo if you want:

countnazgul/enigma-mixin

Set of Enigma mixin. Contribute to countnazgul/enigma-mixin development by creating an account on GitHub.

GitHubcountnazgul

Hope you liked it!

Stefan

qlbuilder

In general qlbuilder is a tool that communicates with Qlik apps (using the Engine API) and set the script. The script is located on the local machine and can be "stitched" from multiple files.

Why?

Everyone have their own preferred IDE/text editor and we can write own Qlik Sense load scripts there and use include/must_include inside the app to "bring" the script inside the app.

But it will be nice if we can write the scrips locally and upload them directly to the app as it is.

The basic idea is that we will write each Qlik Sense script tab in a separate .qvs file and qlbuilder will combine them (when needed) in a single file and set it's content to the dedicated Qlik Sense app (on dedicated QS environment).

qlbuilder can perform few more actions, apart from combining and setting the script: check for syntax errors, reload app, "watching" for script changes and check for syntax errors etc.

With this approach I can write my script in Visual Studio Code (in combination with Qlik for Visual Studio Code
extension) and use all VS Code shortcuts, themes etc.

Installation

qlbuilder is a Node JS module and can be installed as a global package

npm install -g qlbuilder

Folder structure

The folder structure used from qlbuilder looks like this:

project-name
│   config.yml
│───dist
│      LoadScript.qvs
└───src
    │   0--Main.qvs
    │   1--Config.qvs
    │   2--Mapping.qvs
    │   ...

src is the main folder. In this folder are located all qvs files. Each file will result in a separate script tab, when uploaded. The .qvs files are combined in alphabetical order (for now. please have a look at the Road map section at the end of this post)

Config files

There are two configuration files that are used from qlbuilder:

config.yml

This file is mandatory and it's created during the project/folder creation. This file is located in the root folder and defines the Qlik Sense environments and apps ID that are relevant to this script.

The file is in yaml format and looks like this:

- name: dev
  host: 192.168.0.100 # IP/FQDN of QS engine (central node) 
  secure: false # default value is true 
  appId: 12345678-1234-1234-1234-12345678901
  authentication:
    type: winform

The above config defines:

• QS environment called dev
• the environment's host is 192.168.0.100
• the environment is access through http
• the appId (that this script belongs to) is 12345678-1234-1234-1234-12345678901
• the authentication for this environment is Windows/Forms - winform

(for more information on authentication types and other possible settings please have a look at qlbuilder GitHub repo)

The config contains the "public" environment information and can be commit to git repo.

.qlbuilder.yml

The non-public info (authentication information) is held in another configuration file .qlbuilder.yml (there is option to use environment variables as well)

This file is not created through the install/create process and should be located in current user home folder. For example: C:\Users\My-User-Name\.qlbuilder.yml. Using the above example, with winform authentication and environment named dev, the config will looks like:

dev:
  QLIK_USER: DOMAIN\username
  QLIK_PASSWORD: my-secret-password

This information will be used by qlbuilder during the authentication process to generate the sessionId

Commands

qlbuilder is executed from command line and accept few parameters:

• create [project name] - create the initial project folder structure, sample script and config file. If folder with the same name already exists, and the developer agree to re-create, then this command will re-create only the qlbuilder specific files and folders and will not modify any other folders or files

• build - combines all qvs files from src folder into a single load script. The result script is created in dist folder (ideally this file should not be edited)

• checkscript [env name] - check the script for syntax errors. The check is made against temp (session) Qlik app

• getscript [env name] - gets the script from the app id (specified in config.yml) and splits it into multiple qvs files (in src folder). This operation deletes all files in src folder as first step!

• reload [env name] - reloads the QS app and output the reload progress. (the reload is performed on the QS Engine itself)

• setscript [env name] -

  • builds the script from the files in src folder
  • checks the script for syntax errors (if there are errors the process is stopped)
  • opens the QS app and replaces the script with the build one
  • saves the app

• watch [env name] - now this is a funny one :) ... when started in watch mode qlbuilder will check for syntax error on each file save (any qvs file in src folder). During watch mode few sub-commands that can be used:

  • s or set - sets the script to the app and saves it
  • r or rl - reloads the app (if successful the app is saved)
  • c or cls - clears the console
  • x - exit qlbuilder
  • ? - outputs these commands again

Version Control

Having the script locally allows us to put the project folder under version control and keep history of the script changes (anyone thinking about git hooks as well? 😉)

Road map

More or less qlbuilder includes all the features I had in mind initially. But there are few things that can be added:

• include / must_include - (optional config) detect if these statements are present in the script. If found - get the content of the included scripts and replace the statement with the actual script. This can "save" apps from failing because of external script being edited after the app is published
• different way to organize script files? - at the moment the script files are combined based on alphabetical order. It might be a bit annoying when adding new file in the middle of the script - all files that follow need to be renamed

For more information please have a read at the readme at the qlbuilder GitHub repo

Fell free to open issues there if you experiencing any problems or have requests/suggestions

Any suggestions are very welcome!

Hope you liked it!
Stefan

Deploy Qlik extension/mashup with Qlik-CLI

Creating Qlik extensions/mashups is usually quite fun. But it will be better if you have a way to develop locally and deploy to Qlik.

Usually I'm using qExt or sense-go but in this specific case I didn't had the chance to use either (enterprise restrictions)

Luckily for me Qlik-CLI was available. And Qlik-CLI is doing a great job dealing with Qlik Repository API.

I've decided to write a small PowerShell script that can be used to import the extension from my local PC to Qlik.

The script will perform the following actions when started:

• generate wbfolder.wbl file - this is not mandatory but without this file the extension files will not be available for edit in dev-hub
• compress the src folder and place the archive into the build folder
• connect to Qlik
• check if extension with the same name already exists and if there is one - delete it (assume this is an older version)
• import the archive from build folder

After the script is done the new extension/mashup code will be available.

Folder structure

The script assume the following folder structure (need to exists before run)

root
│
└──+ build
│    └─── extension-name.zip (auto-generated and uploaded)
│   
└──+ src
│    │───+ static (optional)
│    │     │─── image.png
│    │     └─── ...
│    │      
│    │─── extension-name.css
│    │─── extension-name.js
│    │─── extension-name.qext
│    │─── wbfolder.wbl
│    └─── ...
│
└─── upload.ps1

Usage

• copy the code from below and create upload.ps1 file in your root folder (the file name can be anything).
• edit the initial variables ($extensionName and $qEnv) to match your extension name and QS DNS
• every time when need to upload the extension/mashup just start the upload.ps1

Summary

Probably not the most elegant way to achieve this but might help if you need quick and "dirty" way to make (or edit exiting one) extension/mashup and use your text editor/IDE of choice.

And since this approach is using PowerShell you can extend it to your needs ;)

PowerShell Script

$extensionName = "mashup-extension-name"
$qEnv = "computerName"

function generateWBL {
    # remove the existing wbfolder.wbl (if exists)
    Remove-Item .\src\wbfolder.wbl -ErrorAction Ignore

    # list all files under the src folder and add them to the wbl file
    $existingFiles = Get-ChildItem -Path .\src\ -File |
        foreach {$_.name} |
        Out-File -FilePath .\src\wbfolder.wbl
}

function compressFolder{
    # archive the content of the src folder
    Compress-Archive -Path .\src\* -DestinationPath .\build\$extensionName.zip -Force
    Write-Host ZIP file created
}

function connectQlik {
    ## connect to Qlik using Qlik-CLI
    Connect-Qlik -computerName "$qEnv" -TrustAllCerts
    Write-Host Connected to Qlik
}

function removeOldExtVersion {
    # Query Qlik for extension with the same name
    $ext = Get-QlikExtension -Filter "name eq '$extensionName'" | Measure-Object

    # if there is such extension - delete it
    if ( $ext.Count -gt 0) {
        Remove-QlikExtension -ename "$extensionName"
        Write-Host Older version found and removed
    }
}

function importExtension {
    # Import the archive for the build folder
    Import-QlikExtension -ExtensionPath .\build\$extensionName.zip
    Write-Host New version was imported
}

function main {
  generateWBL
  compressFolder
  connectQlik
  removeOldExtVersion
  importExtension
  
  Write-Host DONE
}

main

With multi-stage Qlik environment (DEV, TEST, PROD etc) the load scripts are usually including some kind of a check - "In which environment am I?".

Based on the result different/additional actions might be performed. For example:

• if the environment is not PROD then mask/scramble the client names
• use different data connection names (providers) on each environment
• ...

It will be nice if there is something more centralised that can provide this information (and not only) to any app on the cluster. Something like a global meta-data "storage" that can be changed/updated (by the administrators)

In general my approach is to use a dedicated Custom Property and each value to be in format key:pair. Values can be accessed for reading using a (dedicated) REST data connection in the beginning of each script. If we keep the keys the same then we can write one script and use it on every environment.

Custom Property

We will create new custom property (in my case I've named it ENVIRONMENT) and all the global values will be added to this CP.

In QMC:

• create new custom property
• add as many CP values as you want as key:value pairs (I've picket : as a separator but this can be anything). For example:
  • name:DEV
  • central repository ip:192.168.1.10
  • central repository host:qlik.central.repo

Data Connection

We will use the Repository Service REST API to get the ENVIRONMENT custom property values. We need to create a new data connection that will be used to extract the data.

During the installation QS creates few monitor_apps_REST_* data connection which are used by the documents in the Monitoring apps stream (like License Monitor or Operations Monitor). We can use one of these connectors, as a base and create new one that will return the values only for our custom property.

• open any application and create new Data Connection (at this point there is no need the connection to be a specific type. We are creating just a placeholder)
• In QMC -> Data Connection -> monitor_apps_REST_task -> copy the Connection string
• go back and edit the placeholder connection (in my case Environment)
• replace the connection string with the one from monitor_apps_REST_task and apply the change
• go back to the application and edit the Environment connection
• in the URL replace /task/ with /custompropertydefinition/
• in the Query parameters section add new one:
  • Name: filter
  • Value: name eq 'ENVIRONMENT'
• Save

At this point you can preview the data from the connection and if everything is ok you should see something like this:

Usage

Once all is set we can create a small QS script that gets the raw values and "convert" them to a more readable QS format in form of a table with two columns: ENV_NAME and ENV_VALUE

LIB CONNECT TO 'Environment';

RestConnectorMasterTable:
SQL SELECT 
  "__KEY_root",
  (SELECT 
    "@Value",
    "__FK_choiceValues"
   FROM "choiceValues" FK "__FK_choiceValues" ArrayValueAlias "@Value")
FROM JSON (wrap on) "root" PK "__KEY_root";

ENVIRONMENT:
Load
  subfield([@Value], ':', 1) as ENV_NAME,
  subfield([@Value], ':', 2) as ENV_VALUE
;
Load
  [@Value],
  [__FK_choiceValues] as [__KEY_root]
RESIDENT 
  RestConnectorMasterTable
WHERE 
  NOT IsNull([__FK_choiceValues]);

DROP TABLE RestConnectorMasterTable;

This script can be placed in the beginning of each app OR even better - host the script somewhere and each app (on each environment) can Must_Include it.

Upon reload only one table is returned:

Conclusion

Having a centralised place for a cluster-wide variables is a nice to have. I've seen multiple approaches on how to deal with this: specific document prefixes for each env, flat files in each data folder etc. But the problem there is that there is always enough space for someone to make a mistake (overwrite the flat file when moving data is my favorite)

Another plus for custom property and data connection is that these objects are subject of security rules. Which means that we can grant everyone read access over them but very limited audience can edit them which (might 🤞) lead to less errors and maintenance.

Bonus

If you have/use Qlik-CLI then the PowerShell script below will do the "dirty" setup for you. The script will:

• create new custom property with some pre-defined values
• create new REST data connection using the monitor_apps_REST_task as a template
• edit the new data connection
  • change the URL
  • add the additional query parameter

# QS Central Repo Address
$computerName = "qlik.central.repo" # or localhost if started on the server itself

# Properties for the new data connection
# ideally the user/pass should be the ones
# under which the QS service is running
$dataConnProps = @{
  name = "Environment"
  userName = "domain\username"
  password = "password"
}

# Properties for the new custom property
# Can populate some values when creating it
# "Objects = App" is just to create the CP (cant create it without it from Qlik-CLI)
$customPropProps = @{
  name = "ENVIRONMENT"
  values = @("name:DEV", "central repository ip:192.168.1.10")
  objects = @("App")
}


function createNewDataConnection {
    # Get the existing monitor_apps_REST_task connection details
    $connector = Get-QlikDataConnection -filter "name eq 'monitor_apps_REST_task'"

    # Replace the endoint to point to return custom prop details instead of a task
    # Add one more query parameter which will return data only for the Environment custom prop
    $newConnectionString = $connector.connectionString.
        Replace("/qrs/task/full", "/qrs/custompropertydefinition/full").
        Replace("queryParameters=xrfkey%20000000000000000", "queryParameters=xrfkey%20000000000000000%1filter%2name eq '$($customPropProps.name)'")

    # Create the connection
    New-QlikDataConnection -name $dataConnProps.name -connectionstring $newConnectionString -type "QvRestConnector.exe" -username $dataConnProps.userName
}

function createNewCustomProperty {

    # Create the Environment custom property
    New-QlikCustomProperty -name $($customPropProps.name) -choiceValues $($customPropProps.values) -objectTypes $($customPropProps.objects)
}

function connectToQlik {
    Connect-Qlik -computerName $computerName -TrustAllCerts
}

function main {
    connectToQlik
    createNewCustomProperty
    createNewDataConnection
}

main

Stefan

I like how flexible Qlik Sense is. With its massive API if we want we can re-build the GUI (and not only) from scratch and just use the background services (like the Engine or Repository).

I have some (small) issues with how the current Hub looks and behaves and decided to build something simple that fits my needs.

At the end I've wanted something simple that will provide me with quick search capabilities, few shortcuts and the options to create, delete and duplicate files.

And below is the final result

The code is available in GitHub

Just a search box, list with all the apps (to which I have access) and create new app button.

Searching

For the search easyautocomplete plugin is used.

The search works as any other search box. The search is performed on the QS apps titles

Create new app

Pressing the top right green button will show the create new app modal

Here the name of the new app is specified and which will be the initial screen after the app is created. As developer (and not only) "Data editor" or "Data manager" are the main ones.

Options

Agains each app few shortcuts are present

• to which stream the app belongs
• open the "App Overview"
• open the "Data Manager"
• open the "Data Editor"
• open the "Data Model Viewer"
• Duplicate app (not available in Desktop mode)
• Delete app

Hope you liked it!
Stefan