Path from Vrhpolje to old castle or Tolsto is well trodden. Path past playground to forest or through new settlement.
If past playground after 100m to section with "cliff" on right, probably that's the abyss. There you can go left uphill on marked path to Tolsto gora or along edge of abyss further to old castle.
If through new settlement to old castle come to concrete dam at end of that abyss, over dam to other side and path through grass to forest, in forest come to that part where cyclists on descent turn towards pub under rock to grassy part.