吴恩达when to change dev/test sets and metrics

so this way ,you're giving a much large weight to examples that are pornograpic  so that the error term goes up much  more if the algorithm makes mistake on classify a pornographic image as a cat image,in this example ,you've giving 10 times bigger weight to classify pornographic images correctly.

so think of machine learning as two distinct steps. To use the target anology, the first step is place the target,so define (metric) what you want  to aim and then as a completely separate step,second how to aim accurately or how to shoot at the target.Figure out how to do well on that  metric which might be.for example, changing the cost function J that your neural network is optimizing.

in other words,if  we discover that your dev test set has these very high quality images,but evaluating on this dev test set is not perdictive of how well of your application actually performs,because your application needs to deal with lower quality images,then you should change  your dev test set,so that your data better reflcts the types of your actually need to do well on.