Is there one example of using Expectation Over Transformation in Tensorflow Models?

[anselmoferreira]

Is there any working example of ART with custom transformations in a Tensorflow model?

I would like to attack the following model:

json_file = open('resnet-model-against-reprinting.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model = tf.keras.models.model_from_json(loaded_model_json)
model = tf.keras.models.model_from_json(loaded_model_json)
model.load_weights('weights/resnet-model.h5')

The random operations I would like to do are in the following sequential model.

data_transformation = tf.keras.Sequential([tf.keras.layers.GaussianNoise(stddev=tf.random.uniform(shape=(),minval=1000, maxval=1200)),  tf.keras.layers.experimental.preprocessing.RandomZoom(height_factor=(-0.019,-0.029), width_factor=(-0.019,-0.029), fill_mode='reflect', interpolation='bilinear', seed=None, name=None, fill_value=0.0),])

I couldn't find anything in the documentation. Could anybody help me in adapting such a model and transformations to be used to generate an adversarial image with Expectation Over Transformation with the ART?

[beat-buesser]

Hi @anselmoferreira

I think that's a very good question. Unfortunately, ART's current tools in art.preprocessing.expectation_over_transformation don't have an implementation that supports tf.keras.Sequential. The good news are that I think we should have such an implementation and that it is easy to implement. Basically we would need to create an implementation of art.preprocessing.expectation_over_transformation.tensorflow.EoTTensorFlowV2 () e.g. similar to from art.preprocessing.expectation_over_transformation.natural_corruptions.gaussian_noise.tensorflow.EoTGaussianNoiseTensorFlow but with an argument of accepting an instance of tf.keras.Sequential to sample in method _transform which create a random transformation of a single input sample.

Please let me know if you would be interested to implement this tool! I would be happy to provide support and I think would be useful for many users of ART.

[anselmoferreira]

I would love to do that. Just give me some time and I will contact you!

[beat-buesser]

Sounds great!