Stream All the Things in Athens Big Data Meetup – 28 Feb 2017

I have some good news ๐Ÿ™‚

I am very pleasant to announce you that Dean Wampler, Big Data Architect for fast data products atย Lightbend, will speak about streaming processing concepts, problems and solutions in our Athens Big Data Meetup!!!

The main subject is: Stream All the Things

Lightbend (former Typesafe) is the company behind Akka and Scala.

I am very sure that Dean will give a great talk, as his experience is really amazing.

Event will be held in ALBA college at 19:00 (28/2/2017). The venue has around 110 seats but there is space for people to stand as well (please RSVP).

See you there!!!

I am organizing Athens Big Data Meetup along with Euangelos Linardos and Stavros Kontopoulos.
We are always looking for speakers for our meetups. If you would like to give a talk please contact with me.

Regards,
Adrianos Dadis.
โ€”
Real Democracy requires Free Software

Posted in Athens Big Data Meetup | Tagged , , , , | Leave a comment

Thoughts about Agile Greece Summit 2016

Last Friday, I was at Agile Greece Summit 2016.
It was a great conference with amazing speakers and very well organized.

Last year, I attended Agile Greece Summit (2015) which was great, but I believe recent conference was greater by many aspects.
This year there were about 400+ attenders instead of about 200 last year, which means that companies start to invest on Agile!!!
Speakers were very balanced and there were 2 main tracks well aligned with the sessions.

For my opinion, the most valuable sessions this year were:

  • “Managing for Happiness” (by Jurgen Appelo)
  • “15 Teams, 1 Continuous Delivery Pipeline” (by Fredrik Wendt)
  • “Spotify Running: Lessons learned from building a โ€˜Lean Startupโ€™ inside a big tech company” (by Brendan Marsh)
  • “Improving Agility: Learning from Maersk Line Journey” (by ร–zlem Yรผce)

The other sessions were great too, but these 4 were fantastic ๐Ÿ™‚

Here are 16 Agile rules that help me everyday, which are all inspired from Agile Greece Summit conferences (2015 & 2016):

Agile Advices

There are many more things to say about Agile, but there are many good books and people that can help you further.

Agile coaching is fundamental for teams or companies that are new to Agile world.

Great thanks to all speakers, organizers and volunteers!!!

Best regards,
Adrianos Dadis.

Real Democracy requires Free Software

Posted in Release Management, Software Development | Tagged , , | Leave a comment

Big Data Meetup 206 – Apache Storm – Slides and Demo code

Hi all,

last Tuesday I talked about Stream processing using Apache Storm and Apache Kafka at 4th Athens Big Data Meetup.

I really enjoyed the talk. The audience was really interesting as their questions helped me focus and explain various tricky points on stream processing. Great thanks to all and especially to Konstantinos for his kind invitation and help.

Additional to our talk, there was an interesting academic presentation about Artificial Intelligence agents on backgammon game (and more related games) from Dr. Nikolaos Papahristou. Great ideas that make me rethink about AI from Big Data perspective. Compose AI Agents and Big Data techniques/frameworks seems to apply on some interesting use cases that worth investing some time.

Demo code available on github: sentiment-analysis-storm

Presentation slides available on Slideshare.


Regards,
Adrianos Dadis.
โ€”
Real Democracy requires Free Software

Posted in Big Data, Java, Software Development | Tagged , , | Leave a comment

Speaking about Apache Storm and Apache Kafka at Big Data Meetup 24 May

Hello again.

me and Patroclos Christou, we will speak about Big Data Streaming processing using Apache Storm and Apache Kafka on Tuesday 24 May at 4th Big Data Meetup.

I will be glad to meet you there ๐Ÿ™‚

Adrianos Dadis.
โ€”
Real Democracy requires Free Software

Posted in Big Data, Java | Tagged , , | Leave a comment

Apache Storm: How to configure KafkaBolt with Flux

Flux in a mini framework that can help us define and deploy a Storm topology.

Flux has various wrappers that help you define the required stream(s) and initialize your Bolts and Spouts (using constructor with or without arguments and call custom configuration methods automatically via reflection).

What you only need to use Flux is to add it as dependency in your “pom.xml”, configure it via a single YAML file (check flux examples) and then use it as main class to deploy your topology in a Storm cluster (or as local test).

In order to initialize a KafkaBolt the following steps are needed:

  1. Define a “topicSelector” via “withTopicSelector” method
  2. Define a “kafkaMapper” via “withTupleToKafkaMapper” method
  3. Define a “kafkaProducerProps” via “withProducerProperties” method
  4. Initialize “org.apache.storm.kafka.bolt.KafkaBolt” with above configuration
  5. Include above KafkaBolt as part of a stream

Minimal Flux configuration example for KafkaBolt:

components:
  - id: "stringScheme"
    className: "org.apache.storm.kafka.StringScheme"

  - id: "stringMultiScheme"
    className: "org.apache.storm.spout.SchemeAsMultiScheme"
    constructorArgs:
      - ref: "stringScheme"

  - id: "zkHosts"
    className: "org.apache.storm.kafka.ZkHosts"
    constructorArgs:
      - "localhost:2181"

  - id: "topicSelector"
    className: "org.apache.storm.kafka.bolt.selector.DefaultTopicSelector"
    constructorArgs:
      - "myTopicName"

  - id: "kafkaMapper"
    className: "org.apache.storm.kafka.bolt.mapper.FieldNameBasedTupleToKafkaMapper"

  - id: "kafkaProducerProps"
    className: "java.util.Properties"
    configMethods:
      - name: "put"
        args:
          - "bootstrap.servers"
          - "localhost:9092"
      - name: "put"
        args:
          - "acks"
          - "1"
      - name: "put"
        args:
          - "key.serializer"
          - "org.apache.kafka.common.serialization.StringSerializer"
      - name: "put"
        args:
          - "value.serializer"
          - "org.apache.kafka.common.serialization.StringSerializer" 

bolts:    
  - id: "bolt-kafka"
    className: "org.apache.storm.kafka.bolt.KafkaBolt"
    parallelism: 1
    configMethods:
      - name: "withProducerProperties"
        args: [ref: "kafkaProducerProps"]
      - name: "withTopicSelector"
        args: [ref: "topicSelector"]
      - name: "withTupleToKafkaMapper"
        args: [ref: "kafkaMapper"]

streams:
  - name: "spout --> kafkaBolt"
    from: "spout-1"
    to: "bolt-kafka"
    grouping:
      type: LOCAL_OR_SHUFFLE

For a full working configuration example check this, which can be used like this.

Example command to deploy your topology on Storm:

storm jar target/sentiment-analysis-storm-0.0.1-SNAPSHOT.jar org.apache.storm.flux.Flux --remote --c nimbus.host=192.168.1.200 src/test/resources/flux/topology_kafka.yaml

Flux configuration for KafkaSpout is already described as official flux example.

Flux is a really helpful framework that eliminates required custom code to define and initialize a topology ๐Ÿ™‚

Regards,
Adrianos Dadis
โ€”
Real Democracy requires Free Software

Posted in Big Data, Java, Software Development | Tagged , | Leave a comment

FOSSCOMM 2016 – Big Data with Apache Storm – Presentation and Demo code

FOSSCOMM (2016) conference is over!!!

Presenting Big Data and Apache Storm on FOSSCOMM was a really amazing experience ๐Ÿ™‚

It was a great conference with really interesting presentations and people.
A great thanks to FOSSCOMM team and all volunteers.

Demo code is available on github: storm-demo-fosscomm2016
Our presentation is available on Slideshare.


Regards,
Adrianos Dadis.
โ€”
Real Democracy requires Free Software

Posted in Big Data, Java | Tagged , , , | Leave a comment

Speaking about Big Data and Apache Storm at FOSSCOMM conference 16-17 April

Hello everyone,

this weekend FOSSCOMM conference (Free and Open Source Software Communities Meeting) will be held at University of Piraeus.

Together with Patroclos Christou, I will speak about Big Data Streaming processing using Apache Storm and Apache Kafka (event info) on Sunday 17 April 18:00 .

There are many interesting presentations, check full schedule: Saturday & Sunday

I will be glad to meet you there ๐Ÿ™‚

Adrianos Dadis.
โ€”
Real Democracy requires Free Software

Posted in Big Data, Java | Tagged , | Leave a comment