If you are plotting signals from within ADE, and the display.drf for your PDK has the layers y0-y9 (drawing purpose) with their packet set to a dotted pattern, then that's what will happen (this happens for any schematic signals that are plotted - I wouldn't expect it to affect results of calculator expressions).
You could fix your display.drf or tell ViVA not to use the display.drf settings:
envSetVal("asimenv.plotting" "useDisplayDrf" 'boolean nil)
; use the below if you want to override the other default line styles. ;envSetVal("viva.trace" "lineThickness" 'string "Thick") ;envSetVal("viva.trace" "lineStyle" 'string "dot")
Note that the useDisplayDrf control was only added in IC616 ISR9. That's a while back (October 2014) but you didn't say which version you're using.
Regards,