Get Started

The guide below uses the following repository:

https://github.com/oslabs-beta/xkite

README: Github

System Requirements:

- Latest stable versions of Node.js and NPM installed

- Latest stable versions of docker-compose installed

- Note: You will need Docker daemon running before deploying an xkite configuration.

Overview:

This guide explains how to use our core xkite library, complete with a visual interface that will run in the user's browser on a localhost port. If you would rather install xkite into an existing project, or use your command line without the visual interface, please refer to the adjacent tabs.

1. Clone the repository to your device

a. If using NPX, simply execute the following in your terminal and follow the prompts:

npx create-xkite <directory-name>

Upon completion, your dependencies will already be installed. Simply follow the instructions and in the terminal which will include the following:

cd my-xkite

npm run dev

b. Alternatively, you can also follow these steps to clone the xkite repository:

git clone https://github.com/oslabs-beta/xkite

cd xkite

npm install

npm run dev

2. Launching a Custom Configuration

You should now see the xkite running in your localhost browser on port 3000. You can enter the interface by selecting `Create a Data Pipeline` from the entry screen. You will be navigated to a configuration page where you can create a custom Kafka configuration. You can assign custom ports to your setup components, increase or decrease your broker count, choose a data source and data sink, and more.

When you're ready, go ahead and click 'Submit' to trigger a local deployment. xkite will automatically begin downloading and building the appropriate Docker images for your setup. If this is your first time running xkite, this part may take several minutes.

3. Monitoring Kafka Health

Once the appropriate Docker images in your configuration have been downloaded and built, xkite will automatically detect when your metrics are ready to begin monitoring. Upon detection, you will be automatically be navigated to the metrics page.

Explore the Kafka metrics tabs available to you, partitioned based on category. Consume performance indicators related to your producers, consumers, topics, and more.

4. Monitoring Docker Health

xkite integrates Docker health directly into the application interface, making it easy to view the activity status and port assignments of each Docker image in your configuration. Simply navigate to the `Docker Health` tab within the left-hand navigation bar. You can also pause and restart Docker images directly from this page, or pause and restart all using the button in the top navigation.

5. Testing Your Kafka Configuration

You will find a `Tests` tab in your left-hand navigation. This feature allows you to test your deployment by creating new topics and sending test messages to those topics, as well as querying your KSQL instance directly from the xkite interface.

If you have configured your xkite setup to include KSQL as your data source of choice, you will be presented with the option to create streams using the KSQL REST API. Here are the steps you must take:

Open up Docker dashboard into the `Containers` tab, and navigate to the ksql_cli image
Open up the ksql_cli terminal, and enter the following to create the users table with some preconfigured entries:

$ ksql http://8088 <<EOF
  > RUN SCRIPT /tmp/test.sql;
  > exit
  > EOF

Navigate back to the xkite test page within the `KSQL Streams` tab
Simply enter commands into the SQL Query field - here's an example:

SELECT * FROM users EMIT CHANGES;

6. Export Your Kafka Configuration & Wind Down Docker Images

Finally, once you're finished deploying, testing, and monitoring your Kafka setup, you have the option to export any configuration you deploy by clicking the `Export Config` button back on the /configuration page. Please allow several seconds for your files to be exported, zipped, and downloaded from your browser window. Don't forget to gracefully spin down your Docker images when you're finished by clicking the `Disconnect` button. This will clear out both your active containers and delete any related Docker volumes you created during your session.