locked
How to install custom HBase Coprocessor. RRS feed

  • Question

  • Hi,

    We're currently using our own ambari hadoop nodes with hbase and we currently use a couple of coprocessors, some to handle custom queries via a gRPC API, some to send a message to Kafka after certain inserts/updates.

    These coprocessors are now deployed as custom java jar files on the HBase servers.

    In the documentation of HDInsights, it says that Coprocessors are available, and e.g. the coprocessors of Phoenix can be enabled via the Ambari console.

    But can we also deploy our own coprocessor code in the HDInsight managed nodes?

    Thanks in advance,

    Geert P.

    Monday, May 18, 2020 2:48 PM

Answers

  • Hello,

    If it is at region level only, below will work.  (Coprocessor implementing RegionObserver)

    1. The cluster is already created.
    2. Customer can put the custom coprocessor jar in same storage container as HBase’s  (Say created dir /hbase_cp_jars and copied the jar under that)
    3. Add these coprocessor to HBase table(s) using Java APIs or via shell command
    HTableDescriptor#addCoprocessor(String className, Path jarFilePath, int priority, final Map<String, String> kvs)

    Can keep kvs as null.  The class name is the fully qualified class name (FQCN) of RegionObserver implementation class.  The path to jar file is path where the jar been copied.  (/hbase_cp_jars/mycp.jar).  Priority is applicable when they have more than one cp for a table. This says in which order this should get invoked.

    Use shell command and alter the table

    > alter <Table name>, 'coprocessor' => '<path to jar>|<cp FQCN>|<priority>|arg1=1,arg2=2'

    Note:

    cp –  Co processor

    FQCN – Fully Qualified class name

    Hope this helps. Do let us know if you any further queries.

    ----------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Tuesday, May 26, 2020 4:12 AM

All replies

  • Hello,

    I’m working with the product team and get back to you when I have more information.

    Tuesday, May 19, 2020 6:27 AM
  • Hello,

    Product team requesting the below details:

    Do wants to know whether this coprocessor to be on Table side only (RegionObserver) or you want it to be at HMaster side also (MasterObserver)?

    Wednesday, May 20, 2020 9:44 AM
  • Hello,

    Just checking in if you have had a chance to see the previous response. We need the following information to understand/investigate this issue further.

    Friday, May 22, 2020 1:22 PM
  • Sorry for the delay.

    We only have Table-level RegionObservers, no MasterObservers.

    Friday, May 22, 2020 2:13 PM
  • Hello,

    If it is at region level only, below will work.  (Coprocessor implementing RegionObserver)

    1. The cluster is already created.
    2. Customer can put the custom coprocessor jar in same storage container as HBase’s  (Say created dir /hbase_cp_jars and copied the jar under that)
    3. Add these coprocessor to HBase table(s) using Java APIs or via shell command
    HTableDescriptor#addCoprocessor(String className, Path jarFilePath, int priority, final Map<String, String> kvs)

    Can keep kvs as null.  The class name is the fully qualified class name (FQCN) of RegionObserver implementation class.  The path to jar file is path where the jar been copied.  (/hbase_cp_jars/mycp.jar).  Priority is applicable when they have more than one cp for a table. This says in which order this should get invoked.

    Use shell command and alter the table

    > alter <Table name>, 'coprocessor' => '<path to jar>|<cp FQCN>|<priority>|arg1=1,arg2=2'

    Note:

    cp –  Co processor

    FQCN – Fully Qualified class name

    Hope this helps. Do let us know if you any further queries.

    ----------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Tuesday, May 26, 2020 4:12 AM