Uncategorized

Spark Journal : Cell value with String, should not be trimmed

A very short article and precisely focusing on the problem of trimmed string in the cell value on Zeppelin tool.

For those, who don’t know what zeppelin is, its web interactive tool, which helps to run scala / sql on spark cluster directly.

Recently, we found a simple issue, where team reported, long string values were getting trimmed, while querying in zeppelin.
After a bit of R&D, we found its just a simple config on Livy interpreter, which enables / disables this behavior.

The problem
If we try with simple query on zeppelin, where the string length is big, it will trim the string in the result as shown below.

The solution
After setting the property ” zeppelin.livy.spark.sql.field.truncate” as false through the interpreter (Livy2) settings, This worked like a charm.

For setting this
1. Go to the interpreter
2. Click on EDIT for Livy2 interpreter
3. Adding this property and save, it will ask for a restart while saving.

Next time, when you open your notebook and run the same query, it will work as below.

Hope, you learned something with this small article !!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s