Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

STEdge: Self-training Edge Detection with Multi-layer Teaching and Regularization (2201.05121v2)

Published 13 Jan 2022 in cs.CV

Abstract: Learning-based edge detection has hereunto been strongly supervised with pixel-wise annotations which are tedious to obtain manually. We study the problem of self-training edge detection, leveraging the untapped wealth of large-scale unlabeled image datasets. We design a self-supervised framework with multi-layer regularization and self-teaching. In particular, we impose a consistency regularization which enforces the outputs from each of the multiple layers to be consistent for the input image and its perturbed counterpart. We adopt L0-smoothing as the 'perturbation' to encourage edge prediction lying on salient boundaries following the cluster assumption in self-supervised learning. Meanwhile, the network is trained with multi-layer supervision by pseudo labels which are initialized with Canny edges and then iteratively refined by the network as the training proceeds. The regularization and self-teaching together attain a good balance of precision and recall, leading to a significant performance boost over supervised methods, with lightweight refinement on the target dataset. Furthermore, our method demonstrates strong cross-dataset generality. For example, it attains 4.8% improvement for ODS and 5.8% for OIS when tested on the unseen BIPED dataset, compared to the state-of-the-art methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. J. Li, Z. Wang, Z. Pan, Q. Liu, and D. Guo, “Looking at boundary: Siamese densely cooperative fusion for salient object detection,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
  2. G. Zhu, J. Li, and Y. Guo, “Supplement and suppression: Both boundary and nonboundary are helpful for salient object detection,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
  3. M. Li, D. Chen, S. Liu, and F. Liu, “Semisupervised boundary detection for aluminum grains combined with transfer learning and region growing,” IEEE Transactions on Neural Networks and Learning Systems, 2021.
  4. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 5, pp. 898–916, 2010.
  5. C. Rother, V. Kolmogorov, and A. Blake, “” grabcut” interactive foreground extraction using iterated graph cuts,” ACM transactions on graphics (TOG), vol. 23, no. 3, pp. 309–314, 2004.
  6. C. Li, W. Xia, Y. Yan, B. Luo, and J. Tang, “Segmenting objects in day and night: Edge-conditioned cnn for thermal image semantic segmentation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 7, pp. 3069–3082, 2020.
  7. B. Zhao and X. Li, “Edge-aware network for flow-based video frame interpolation,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  8. K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi, and M. Ebrahimi, “Edgeconnect: Generative image inpainting with adversarial edge learning,” arXiv preprint arXiv:1901.00212, 2019.
  9. F. Fang, J. Li, Y. Yuan, T. Zeng, and G. Zhang, “Multilevel edge features guided network for image denoising,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 9, pp. 3956–3970, 2020.
  10. J. Canny, “A computational approach to edge detection,” IEEE Transactions on pattern analysis and machine intelligence, no. 6, pp. 679–698, 1986.
  11. P. Dollár and C. L. Zitnick, “Fast edge detection using structured forests,” IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 8, pp. 1558–1570, 2014.
  12. S. Xie and Z. Tu, “Holistically-nested edge detection,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1395–1403.
  13. Y. Liu, M.-M. Cheng, X. Hu, K. Wang, and X. Bai, “Richer convolutional features for edge detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3000–3009.
  14. J. He, S. Zhang, M. Yang, Y. Shan, and T. Huang, “Bi-directional cascade network for perceptual edge detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3828–3837.
  15. X. S. Poma, E. Riba, and A. Sappa, “Dense extreme inception network: Towards a robust cnn model for edge detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020, pp. 1923–1932.
  16. Z. Su, W. Liu, Z. Yu, D. Hu, Q. Liao, Q. Tian, M. Pietikäinen, and L. Liu, “Pixel difference networks for efficient edge detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5117–5127.
  17. M. Pu, Y. Huang, Y. Liu, Q. Guan, and H. Ling, “Edter: Edge detection with transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1402–1412.
  18. D. A. Mély, J. Kim, M. McGill, Y. Guo, and T. Serre, “A systematic comparison between visual cues for boundary detection,” Vision research, vol. 120, pp. 93–107, 2016.
  19. G. French, T. Aila, S. Laine, M. Mackiewicz, and G. Finlayson, “Semi-supervised semantic segmentation needs strong, high-dimensional perturbations,” 2019.
  20. J. Kim, J. Jang, and H. Park, “Structured consistency loss for semi-supervised semantic segmentation,” arXiv preprint arXiv:2001.04647, 2020.
  21. L. Xu, C. Lu, Y. Xu, and J. Jia, “Image smoothing via l 0 gradient minimization,” in Proceedings of the 2011 SIGGRAPH Asia conference, 2011, pp. 1–12.
  22. O. Chapelle and A. Zien, “Semi-supervised classification by low density separation,” in International workshop on artificial intelligence and statistics.   PMLR, 2005, pp. 57–64.
  23. Y. Grandvalet and Y. Bengio, “Semi-supervised learning by entropy minimization.” CAP, vol. 367, pp. 281–296, 2005.
  24. J. Kittler, “On the accuracy of the sobel edge detector,” Image and Vision Computing, vol. 1, no. 1, pp. 37–42, 1983.
  25. G. Bertasius, J. Shi, and L. Torresani, “Deepedge: A multi-scale bifurcated deep network for top-down contour detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 4380–4389.
  26. W. Shen, X. Wang, Y. Wang, X. Bai, and Z. Zhang, “Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3982–3991.
  27. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  28. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition.   Ieee, 2009, pp. 248–255.
  29. J. K. Wibisono and H.-M. Hang, “Fined: Fast inference network for edge detection,” arXiv preprint arXiv:2012.08392, 2020.
  30. F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258.
  31. H. Scudder, “Probability of error of some adaptive pattern-recognition machines,” IEEE Transactions on Information Theory, vol. 11, no. 3, pp. 363–371, 1965.
  32. S. Fralick, “Learning to recognize patterns without a teacher,” IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 57–64, 1967.
  33. D.-H. Lee et al., “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in Workshop on challenges in representation learning, ICML, vol. 3, no. 2, 2013, p. 896.
  34. L.-C. Chen, R. G. Lopes, B. Cheng, M. D. Collins, E. D. Cubuk, B. Zoph, H. Adam, and J. Shlens, “Naive-student: Leveraging semi-supervised learning in video sequences for urban scene segmentation,” in European Conference on Computer Vision.   Springer, 2020, pp. 695–714.
  35. B. Zoph, G. Ghiasi, T.-Y. Lin, Y. Cui, H. Liu, E. D. Cubuk, and Q. V. Le, “Rethinking pre-training and self-training,” arXiv preprint arXiv:2006.06882, 2020.
  36. Y. Zhu, Z. Zhang, C. Wu, Z. Zhang, T. He, H. Zhang, R. Manmatha, M. Li, and A. Smola, “Improving semantic segmentation via self-training,” arXiv preprint arXiv:2004.14960, 2020.
  37. W.-C. Hung, Y.-H. Tsai, Y.-T. Liou, Y.-Y. Lin, and M.-H. Yang, “Adversarial learning for semi-supervised semantic segmentation,” arXiv preprint arXiv:1802.07934, 2018.
  38. M. S. Ibrahim, A. Vahdat, M. Ranjbar, and W. G. Macready, “Semi-supervised semantic image segmentation with self-correcting networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12 715–12 725.
  39. X. Chang, P. Ren, P. Xu, Z. Li, X. Chen, and A. Hauptmann, “A comprehensive survey of scene graphs: Generation and application,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 1–26, 2021.
  40. L. Zhang, X. Chang, J. Liu, M. Luo, Z. Li, L. Yao, and A. Hauptmann, “Tn-zstad: Transferable network for zero-shot temporal activity detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  41. M. Li, P.-Y. Huang, X. Chang, J. Hu, Y. Yang, and A. Hauptmann, “Video pivoting unsupervised multi-modal machine translation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  42. C. Yan, X. Chang, Z. Li, W. Guan, Z. Ge, L. Zhu, and Q. Zheng, “Zeronas: Differentiable generative adversarial networks search for zero-shot learning,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 12, pp. 9733–9740, 2021.
  43. Y. Li, M. Paluri, J. M. Rehg, and P. Dollár, “Unsupervised learning of edges,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1619–1627.
  44. Y. Ouali, C. Hudelot, and M. Tami, “Semi-supervised semantic segmentation with cross-consistency training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12 674–12 684.
  45. Z. Ke, D. Wang, Q. Yan, J. Ren, and R. W. Lau, “Dual student: Breaking the limits of the teacher in semi-supervised learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 6728–6736.
  46. X. Chen, Y. Yuan, G. Zeng, and J. Wang, “Semi-supervised semantic segmentation with cross pseudo supervision,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2613–2622.
  47. A. Ghosh, H. Kumar, and P. Sastry, “Robust loss functions under label noise for deep neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017.
  48. A. Ghosh, N. Manwani, and P. Sastry, “Making risk minimization tolerant to label noise,” Neurocomputing, vol. 160, pp. 93–107, 2015.
  49. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision.   Springer, 2014, pp. 740–755.
  50. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, pp. 8026–8037, 2019.
  51. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  52. J. Revaud, P. Weinzaepfel, Z. Harchaoui, and C. Schmid, “Epicflow: Edge-preserving interpolation of correspondences for optical flow,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1164–1172.
  53. F. Galasso, N. S. Nagaraja, T. J. Cardenas, T. Brox, and B. Schiele, “A unified video segmentation benchmark: Annotation, metrics and analysis,” in Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 3527–3534.
  54. A. Prest, C. Leistner, J. Civera, C. Schmid, and V. Ferrari, “Learning object class detectors from weakly annotated video,” in 2012 IEEE Conference on computer vision and pattern recognition.   IEEE, 2012, pp. 3282–3289.
  55. X. Ren and L. Bo, “Discriminatively trained sparse code gradients for contour detection,” in Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 1, 2012, pp. 584–592.
  56. P. Isola, D. Zoran, D. Krishnan, and E. H. Adelson, “Crisp boundary detection using pointwise mutual information,” in European Conference on Computer Vision.   Springer, 2014, pp. 799–814.
  57. S. Hallman and C. C. Fowlkes, “Oriented edge forests for boundary detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1732–1740.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Yunfan Ye (10 papers)
  2. Renjiao Yi (23 papers)
  3. Zhiping Cai (25 papers)
  4. Kai Xu (312 papers)
Citations (5)
View on Semantic Scholar