fix: because lock is not release, drop cutover sentry table is hanged#1180
fix: because lock is not release, drop cutover sentry table is hanged#1180rashiq merged 4 commits intogithub:masterfrom
Conversation
|
@dm-2 |
|
@dm-2 @timvaillancourt |
|
The problem and the fix make sense to me 👍 @timvaillancourt would you mind taking a look as well? 🙇 Your thoughts would be much appreciated, given that it's such a critical section of code! |
|
Hey @dm-2; I just got hit with this bug as well and I would love to get this or some related fix merged -- would you mind if I took a look and did some testing on this PR to validate? |
| } | ||
| }) | ||
| if _, err := tx.Exec(query); err != nil { | ||
| this.migrationContext.Log.Errore(err) |
There was a problem hiding this comment.
It might be better to execute this query with a timeout and exit the program if it fails. While this PR should change ghost so that it will always successfully unlock the tables, in the event that there is some other unforseen race condition or deadlock here, then stopping program execution would be a guaranteed way to at least release all locks held by this program.
It would have the unfortunate side effect of leaving the _del table in existence which would need to be cleaned up later, but IMO that's operationally better than holding table locks which will cause any application actively trying to use the tables to stall.
|
@rashiq @dm-2 I've tested this PR and it looks like it fixes the problem. I recommend that we go ahead and merge this! |
Sorry for the delay, this LGTM 👍 |
|
@rashiq Thanks for the approval! Would you be able to merge this? |
|
@gtowey-air Hey! (also nice to see you again!) - sorry for the delay, merging now |
Related issue: #1171