Not knowing the specifics of the document beyond what you have shared, you have three real options:
1) use tables instead of tabs, and just set the borders to be blank. Then your second column can have bullets and will auto-bullet/indent/whatever. this also has the benefit of keeping all your column 2 text together even if the line starts to wrap around
2) Same as above, but use columns instead of tables. Columns are a little more tricky to control though, in terms of where they start and stop. But only a little.
2) use Insert -> Symbol instead of the bullet feature to add bullets wherever you want.What you won't get is the correct indenting if the line comes back
I would NOT use text boxes, as those can be a real nightmare when you move from version to version or machine to machine.