Hii,
Use t-if + group check
To fully remove the button for internal users please try below code:
<template id="cw_sale_order_portal_content_inherit" inherit_id="sale.sale_order_portal_content">
<xpath expr="//span[@id='shipping_address']/h5" position="replace">
<h5 class="mb-1">Delivery Address</h5>
</xpath>
<xpath expr="//a[contains(normalize-space(text()), 'Accept & Sign')]" position="replace">
<t t-if="not request.env.user.has_group('base.group_user')">
<a href="/my/orders/accept" class="btn btn-primary">Accept & Sign</a>
</t>
</xpath>
</template>
base.group_user → This is the internal user group.
So not request.env.user.has_group('base.group_user') means: show this only to non-internal users → portal or public .
i hope it is use full