VK Cloud logo
Updated at April 15, 2024   08:50 AM

Quick startBeta

A quick start will help you get started with the service and get to know its capabilities.

After completing all the steps of a quick start, you will:

  1. Create an instance of Cloud Kafka in the Single configuration.
  2. Learn how to send messages to a topic and receive messages from the topic of the Cloud Kafka instance.

1. Create an instance of the Cloud Kafka service

  1. Go to VK Cloud personal account.

  2. Select the project where you want to create an instance of the service.

  3. Go to Data Platform → Экземпляры сервисов.

  4. Click the Создать экземпляр button.

  5. At the “Configuration” step:

    1. Select the type of service Kafka.
    2. Select version 3.5.0.
    3. Select the configuration Single.
    4. Click the Next step button.
  6. At the “Parameters” step:

    1. Set a name and description for the instance: vkcloud-demo-kafka and The first instance of Cloud Kafka.

    2. Set the node parameters for the broker:

      • Type of virtual machine: Standard-2-4.
      • Disk Type: High-IOPS SSD.
      • Disk size: 50 ГБ.
    3. Кластер Kubernetes: choose Создать новый кластер.

    4. Network: choose Create new network.

    5. Choose Neutron as SDN.

    6. Subnet address: type 10.0.1.0/24.

    7. Availability zone: Москва (GZ1).

    8. Click the Next step button.

  7. At the “Учетные данные” step:

    1. Логин администратора для доступа к Kafka: for example, kafkaadmin.

    2. Пароль администратора для доступа к Kafka: click the Generate button and save the received password.

    3. Click the Create button.

      Wait for the operation to complete. It may take a long time to create an instance of Cloud Kafka.

2. Prepare to work with a Cloud Kafka instance

  1. Get access to Kafka console clients on the host from which you plan to work with the service instance:

    1. Make sure that Java version 11 or higher is installed on the host:

      java --version

      If the output contains Command 'java'not found or the version is outdated, install the Java Development Kit (JDK), for example, OpenJDK. You can install both the entire JDK and only the Java Runtime Environment (JRE).

    Example of installing OpenJDK 11 (JRE only) using apt for Ubuntu 22.04 LTS:

    sudo apt update && sudo apt install openjdk-11-jre
    1. Download the archive with the Kafka distribution version 3.5.0. The Kafka versions for the service instance and on the host must match. Example of a command to download a distribution using cURL:

      curl -O https://archive.apache.org/dist/kafka/3.5.0/kafka_2.13-3.5.0.tgz
    2. Unzip the archive and go to the directory with the unpacked files:

      tar -xzf kafka_2.13-3.5.0.tgz && cd kafka_2.13-3.5.0
    3. Get the version of the producer console client and the consumer console client:

      bin/kafka-console-producer.sh --versionbin/kafka-console-consumer.sh --version

      Successful output of the version indicates that the clients are working correctly.

    4. Create a configuration file client-ssl.properties with the contents:

      security.protocol=SSL

      This file contains the setting, which requires the producer client and the consumer client to authenticate to the service instance using the SSL protocol.

  2. Get the address of the bootstrap server to connect to the service instance:

    1. Go to VK Cloud personal account.
    2. Select the project where the service instance is located.
    3. Go to Data Platform → Экземпляры сервисов.
    4. Open the vkcloud-demo-kafka cluster page by clicking on its name.
    5. Go to the General information tab.
    6. Write down the address from the Bootstrap parameter.

3. Send a message to the topic and receive it

  1. Create a separate terminal session in which the console client will work.

  2. Run the client-consumer in this session and specify the test topic as the source. The client-consumer will connect to the Cloud Kafka instance using the previously received bootstrap server address and configuration file, and continuously read and output messages from the topic. A new topic will be created automatically the first time you access it.

    To do this, run the command:

    bin/kafka-console-consumer.sh \  --bootstrap-server kafka-bootstrap.example.com:9093 \  --consumer.config client-ssl.properties \  --topic test
  3. Create another terminal session in which another console client will be running.

  4. Run the client-producer in this session and specify the test topic as the receiver. The client-producer will connect to the Cloud Kafka instance using the previously received bootstrap server address and configuration file, and send the input messages to the topic.

    To do this, run the command:

    bin/kafka-console-producer.sh \  --broker-list kafka-bootstrap.example.com:9093 \  --producer.config client-ssl.properties  --topic test
  5. In the terminal session with the manufacturer client, enter a test message and then press Enter:

    This is the test message.
  6. Check that the sent test message is displayed in the terminal session with the consumer client:

    This is the test message.

    The message means that the Cloud Kafka instance works correctly.

  7. Shut down the clients by pressing the keyboard shortcut Ctrl + C in both terminal sessions.

Delete unused resources

The instance of Cloud Kafka is charged and consumes computing resources. If you don't need it anymore, delete it.

You can also delete the configuration file client-ssl.properties and the downloaded Kafka distribution.