Skip to content

Comments

Fix Spark parameter creation when passing a nil-value named parameter to a query #199

Merged
kravets-levko merged 9 commits intodatabricks:mainfrom
esdrasbeleza:main
Apr 16, 2024
Merged

Fix Spark parameter creation when passing a nil-value named parameter to a query #199
kravets-levko merged 9 commits intodatabricks:mainfrom
esdrasbeleza:main

Conversation

@esdrasbeleza
Copy link
Contributor

@esdrasbeleza esdrasbeleza commented Apr 1, 2024

  • When a sql.NamedValue has the field Value set to nil, the resulting cli_service.TSparkParameter will also have the value nil instead of *cli_service.TSparkParameterValue{StringValue: *"%!s(<nil>"}.
  • Add the type SqlVoid, following the conventions used in the NodeJS connector and the Python driver.

Fix #193.

Copy link
Contributor

@kravets-levko kravets-levko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Though, better if someone of codeowners make a final decision

@kravets-levko
Copy link
Contributor

@esdrasbeleza Can you please rebase and resolve conflicts? Thanks!

esdrasbeleza and others added 9 commits April 15, 2024 12:34
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
![image](https://github.com/databricks/databricks-sql-go/assets/12139186/7e111bdb-3644-431e-aeb4-c4a8d92787a6)

---------

Signed-off-by: Levko Kravets <levko.ne@gmail.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
`os.ReadFile` reads all of the content of the file into a byte array in
memory, which can cause memory consumption pressure for users. Instead,
an `os.File` instance is itself a byte reader, and we can provide the
file directly to `http.NewRequest` so it can read the file in chunks and
upload it as a stream, thus not holding the whole file in memory.

Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Closes #204

Signed-off-by: candiduslynx <candiduslynx@gmail.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Closes #206

Signed-off-by: candiduslynx <candiduslynx@gmail.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Signed-off-by: Levko Kravets <levko.ne@gmail.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Attempt to fix #214

Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
Signed-off-by: Esdras Beleza <esdras@esdrasbeleza.com>
@esdrasbeleza
Copy link
Contributor Author

@kravets-levko got a bit lost in the process with merges, rebases, signoffs etc., but it's done 😅
10+ years of git and it's never enough!

@esdrasbeleza
Copy link
Contributor Author

@kravets-levko any plans to merge this? My current code base is full of hacky queries because of this issue 🙏

@kravets-levko kravets-levko merged commit aeb5e5d into databricks:main Apr 16, 2024
@kravets-levko
Copy link
Contributor

@esdrasbeleza Merged 👍 I'd like to ship one more PR with this release - waiting for a person to rebase their branch and we're good to go. After that I'll cut a release ASAP

atzoum pushed a commit to rudderlabs/sqlconnect-go that referenced this pull request May 20, 2024
….5.4 to 1.5.5 (#66)

Bumps
[github.com/databricks/databricks-sql-go](https://github.com/databricks/databricks-sql-go)
from 1.5.4 to 1.5.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sql-go/releases">github.com/databricks/databricks-sql-go's
releases</a>.</em></p>
<blockquote>
<h2>v1.5.5</h2>
<ul>
<li>Fix: handle
<code>nil</code><code>databricks/databricks-sql-go#199</code><a
href="https://github.com/esdrasbeleza"><code>@​esdrasbeleza</code></a>)</li>
<li><code>databricks/databricks-sql-go#217</code><a
href="https://github.com/candiduslynx"><code>@​candiduslynx</code></a>)</li>
<li><code>databricks/databricks-sql-go#215</code><a
href="https://github.com/esdrasbeleza"><code>@​esdrasbeleza</code></a>)</li>
<li><code>databricks/databricks-sql-go#210</code><a
href="https://github.com/tubiskasaroos"><code>@​tubiskasaroos</code></a>)</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/databricks/databricks-sql-go/compare/v1.5.4...v1.5.5">https://github.com/databricks/databricks-sql-go/compare/v1.5.4...v1.5.5</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sql-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sql-go's
changelog</a>.</em></p>
<blockquote>
<h2>v1.5.5 (2024-04-16)</h2>
<ul>
<li>Fix: handle
<code>nil</code><code>databricks/databricks-sql-go#199</code><a
href="https://github.com/esdrasbeleza"><code>@​esdrasbeleza</code></a>)</li>
<li><code>databricks/databricks-sql-go#217</code><a
href="https://github.com/candiduslynx"><code>@​candiduslynx</code></a>)</li>
<li><code>databricks/databricks-sql-go#215</code><a
href="https://github.com/esdrasbeleza"><code>@​esdrasbeleza</code></a>)</li>
<li><code>databricks/databricks-sql-go#210</code><a
href="https://github.com/tubiskasaroos"><code>@​tubiskasaroos</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/697ea4fc9c487676f06b36e53212c9e971a07d36"><code>697ea4f</code></a>
Prepare release v1.5.5 (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/219">#219</a>)</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/700d02fdb02864912f900b385ade03849542e949"><code>700d02f</code></a>
feat: Update Apache Arrow to <code>v16</code> (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/216">#216</a>)</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/f06515c9c67839b91365fa1def480ae5a494ce26"><code>f06515c</code></a>
Fixed the AzureTenantId in oauth.go (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/210">#210</a>)</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/aeb5e5d885adfb4a4caf3dc2008c03d46a8da8de"><code>aeb5e5d</code></a>
Fix Spark parameter creation when passing a <code>nil</code>-value named
parameter to a ...</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/4f9a1a11d657c70c1f8b520cfd4e5e8656f71ccf"><code>4f9a1a1</code></a>
hotfix: Provide content length on file put (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/217">#217</a>)</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/42b1542e423300912920b7ad16eabd2dd0236ed7"><code>42b1542</code></a>
Add Kravets Levko to codeowners (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/218">#218</a>)</li>
<li><a
href="https://github.com/databricks/databricks-sql-go/commit/7c4ada8ee7fbd260684c0b37e6606d609a32facc"><code>7c4ada8</code></a>
Fix formatting of *float64 parameters (<a
href="https://redirect.github.com/databricks/databricks-sql-go/issues/215">#215</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sql-go/compare/v1.5.4...v1.5.5">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/databricks/databricks-sql-go&package-manager=go_modules&previous-version=1.5.4&new-version=1.5.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue while converting nil SQL parameter to null

4 participants